It is said that a picture is worth a thousand words – and, according to Forrester Research, a minute of video may be worth 1.8 million words. For businesses ranging from ecommerce to social media, visual content is worth more than the amount of words that it conveys: it is an opportunity to build customer engagement, increase trust and safety, enhance personalization, and glean actionable insights based on content engagement.

Coactive, a visual data analytics startup founded by CEO Cody Coleman and Will Gaviria Rojas in 2021, is democratizing the opportunity for businesses to analyze images and videos.

Coactive’s co-founders Will Gaviria Rojas and Cody Coleman

Images and videos are unstructured data—information that has not yet been ordered in a predefined way—that traditionally require machine learning expertise, a robust technical infrastructure, and significant amounts of time to accurately analyze.

Coactive, the platform built on Amazon Web Services (AWS) and available on the AWS Marketplace, helps data practitioners derive rapid insights from unstructured data at scale and with minimal supervision. Accessible by user interface or APIs, the platform’s capabilities range from intelligent search to production analytics that use the full power of SQL.

Proving what’s possible in AI

Coactive’s innovative solution results from intensive amounts of time, research, and determination. During 2018, while earning his PhD in Computer Science at Stanford, Cody recognized that “artificial intelligence and intelligent applications were going to be the future. The blocker was that you needed hundreds of thousands of dollars’ worth of equipment and tremendous amounts of data to accomplish anything significant.”

Use SQL to run analytics on your visual dataset. This photo shows a SQL query used to categorize images

Bothered by these limitations, Cody committed to lowering the barriers to entry for machine learning so that everyone could benefit from it: “My mission during graduate school was to use my passion for computer science to benefit society at large while serving as a leader for future generations.”

Cody joined the Stanford DAWN research project, a group focused on making it dramatically easier to build AI-powered applications. One of the many impressive breakthroughs from Cody’s work was DAWNBench, the first end-to-end machine learning (ML) systems benchmark used by global technology companies as the industry standard. In its first year, DAWNBench reduced model training time by 500x and training cost by 20x. Galvanized by the progress he’d made in creating accessible AI, Cody tackled the next big question: What to do next?

At this time–by serendipity or coincidence–Cody’s friend Will moved to the San Francisco Bay area to begin a career at a large technology company. With a friendship spanning 10 years since their time as undergrads at Massachusetts Institute of Technology (MIT), Cody helped Will move in. “Will asked me the two questions you should never ask a PhD student because they cause an immediate existential crisis,” laughs Cody. “’When are you going to graduate?’ and ‘What are you going to do after?’”

Cody had considered options that ranged from joining a prestigious technology company, becoming a university faculty member, joining a startup, or building a company. “Without hesitation, Will told me to build my own company,” says Cody. “He told me that it’s the right time and I have the right knowledge. And that he’d love to join me in this journey.”

Months of conversations, research, and studying the problem first-hand led Cody and Will to come to the same realization: People need a visual analytics platform to unlock the value of their content, and it’s time to build it. With that decision, Coactive was founded.

The Coactive team works together on version 1.0 of the Coactive platform.

Creating a visual analytics solution for everyone

Machine learning advanced significantly during Cody’s time at Stanford, but there was still much work to be done to make AI applications accessible to everyone: from the world’s largest companies, to a startup designing their minimum viable product.

This was particularly true about machine learning that analyzed images and videos to derive actionable insights. For these unstructured data formats, the end-to-end workflow could require high-end large-scale compute in the form of GPUs, significant storage capacity, and large amounts of time and expertise due to the complexity of the process. A common workflow may include the following:

Data scientists complete data exploration and build computer vision models to analyze and understand the visual data.
ML engineers operationalize these models.
Software engineers plug the model predictions into real world applications for consumers.

To make visual content analysis more accessible, accurate, and efficient, Coactive pairs the breadth of existing large language models (LLMs) with the accuracy and automation that comes from applying a learning system to domain-specific data. After customers provide access to their large volumes of raw image and video files, Coactive uses pre-trained foundation models in conjunction with their proprietary active learning and classification system to embed and index the data. During this process, customers have the option to upload existing labels or provide a few examples so the Coactive platform can further learn any domain-specific nuances of their data.

“One of the very powerful things about really large models is that we don’t actually need to toss a massive quantity of data to fine-tine for specific tasks,” explains Cody. “They call these large language models ‘few-shot learners’ for a reason. Rather than thinking about the quantity of data we toss at these systems, it’s really about quality.”

The result? Customers can use Coactive to query, search, filter, and analyze visual content rapidly and at massive scale.

Partnering with AWS to accelerate success

As an innovative and rapidly-scaling startup, Coactive decided to migrate from their original cloud provider go all-in on AWS. Solutions offered by AWS align with Coactive’s four primary cloud provider needs: Depth and breadth of services, optionality in tooling, availability to power the scaling of their product, and security-first offerings.

“We needed to build our solution on a cloud provider that could handle enterprise scale while being flexible enough to let us create something entirely new. With AWS, we were able to do this while ensuring best-in-class security to our customers,” says Cody.

Building with AWS solutions

After migration, Coactive set to work building a cutting-edge web application using AWS solutions such as Amazon Simple Storage Service (Amazon S3), Amazon Aurora, and Amazon Elastic Container Service (Amazon ECS). This web application helped Coactive establish their initial MVP and run proof of concepts for prospects.

For their data-centric machine learning jobs, Coactive benefits from using Amazon Aurora PostgreSQL Serverless to serve low latency database requests without having to spend time managing their database infrastructure. Coactive’s many petabytes of image and video data are stored using Amazon S3.

To front their web application, Coactive uses a combination of Amazon CloudFront as its content delivery network (CDN). The backen...

Unlocking the value of unstructured data: How Coactive built a visual analytics platform on AWS