Snowflake launches autonomous data tools & live analytics

Snowflake has introduced new autonomous data engineering and real-time analytics products aimed at companies using AI agents that need live, governed data.

The new tools are designed to automate recurring work in building, maintaining and repairing data pipelines, while leaving final production decisions to engineers. Snowflake is also expanding its real-time analytics offering so businesses can ingest, transform and query live data on the same platform.

At the centre of the update is Snowflake CoCo, the company's coding agent for data teams. New CoCo Cloud Agents let teams run coding sessions in a Snowflake-managed cloud environment rather than on an engineer's laptop, while CoCo Automations lets users schedule recurring tasks such as checking pipeline health or investigating failed jobs.

Engineers can describe a task in plain language, set how often it runs and apply restrictions to control what the agent is allowed to do. If a scheduled run identifies a pipeline problem, CoCo can trace a likely cause and propose a fix for engineers to review before any code moves into production.

Snowflake is pairing those functions with workflow and collaboration features. Snowflake for Slack brings CoCo into messaging channels, while a GitHub integration keeps proposed code changes within existing review processes.

Snowflake is also pushing further into pipeline creation with Snowflake Pipeline Builder, which gives teams a visual editor inside Snowflake Workspaces or the option to ask CoCo to assemble pipeline steps. Customers including auxmoney and New American Funding are already using the product.

"Every AI success story has a data team behind it," said Christian Kleinerman, EVP of Product, Snowflake. "However, even as AI scales across the enterprise, those teams are still keeping data accurate largely by hand. With data volumes continuing to skyrocket and agents taking on more of the work, that approach won't hold. Snowflake is bringing autonomous agents into data engineering and real-time data onto one governed platform, so data teams can stop holding the foundation together and spend more time building on it."

Customer examples

Auxmoney said Pipeline Builder is helping it connect work produced by different teams into end-to-end data pipelines. The European digital lending platform has been moving more of its data work onto Snowflake so it runs where the data already sits.

"As a leading digital-lending platform for consumer credit in Europe, our risk and analytics teams depend on data that is accurate and well governed," said Marius Garbowski, Vice President of Data & Analytics, auxmoney. "We've been bringing more of that work onto Snowflake, so it runs where our data already lives, and Pipeline Builder helps us connect the steps our teams build into end-to-end pipelines, so new insights improve our models and reach the business sooner. Instead of piecing together our own tooling, we can rely on Snowflake to provide a robust foundation, allowing our teams to focus even more on data science and models that move our business forward."

New American Funding described the tools as part of a broader modernisation of legacy pipelines and reports.

"Snowflake has changed what our data team can deliver for the business," said Dian Xu, VP of Engineering, New American Funding. "Snowflake CoCo has taken on much of the heavy lifting as we modernise hundreds of legacy pipelines and reports. Pipeline Builder is taking it a step further to give us a consistent way to turn that work into pipelines we can rely on every day. This allows our engineers to spend more time on the products that help borrowers and loan officers."

Real-time focus

Alongside the data engineering products, Snowflake outlined a broader real-time analytics push. Businesses often stitch together multiple tools to move live data into dashboards, apps and internal systems, adding cost and governance complexity.

Snowflake's approach brings ingestion, transformation and serving together. Snowpipe Streaming is positioned as the entry point for data from apps and devices, while Datastream is intended to let companies already using Apache Kafka send application data into Snowflake without managing separate brokers or connectors. Streaming Transformations then applies SQL-based processing as data arrives.

The aim is to make live data available to downstream systems within seconds. Snowflake said businesses can stream up to 20 GB of data per second into the platform, with a median data-readiness time of three seconds from ingestion through transformation to analytics.

Interactive Analytics is intended to support customer-facing dashboards, applications and AI agents directly from Snowflake. The company said this should reduce the need for a separate serving database while keeping data governed from the point of ingestion.

Customers cited by Snowflake include Minted, Nexon and Cboe Global Markets. Cboe said it now brings almost 200 billion rows of market data into Snowflake each day using Snowpipe Streaming.

"Our global trading markets run on massive volumes of data. Our participants expect it to be accurate and available almost instantly, when they need it," said Kevin Knapp, Senior Director of Software Engineering, Cboe Global Markets. "Our teams used to wait until the end of the day to analyse and deliver actionable reporting on market data because our legacy system moved it in batches. With Snowpipe Streaming, we now bring almost 200 billion rows of market data into Snowflake every day, and our teams can query it within seconds or minutes. Real-time access to market insights and customer trading behavior on a Snowflake platform that has proven to deliver cloud scalability and reliability is critical to our business operations."

Spark migration

Snowflake also used the launch to target organisations running Apache Spark workloads. Updates to Snowpark Connect and related tools are intended to let customers move Spark pipelines that write to Apache Iceberg tables onto Snowflake with fewer code changes, while a file reader for Apache Spark allows direct loading of JSON, CSV and Parquet files.

Snowflake said CoCo can review existing Spark code and rewrite it to run on Snowflake, while a monitoring interface helps teams track job performance and identify failures. Klarent said moving heavy Spark workloads to Snowflake cut its total cost of ownership by 60 per cent and reduced the runtime of its longest pipeline from more than 10 hours to under three.

"Moving our heavy Spark workloads to Snowflake cut our total cost of ownership by 60 percent," said Kumar Maddali, VP of Engineering and AI, Klarent. "Our longest pipeline went from more than 10 hours to under three, and our engineers no longer spend their time keeping clusters running. We now have a simpler, faster foundation for our data, and our team can focus on building."