Symphony RetailAI helps retailers and CPG manufacturers drive profitable revenue growth through AI-enabled decision-making. Its customers include 15 of the world’s 25 largest grocery retailers, thousands of retail brands, and hundreds of national and regional chains, who rely on Symphony RetailAI to transform their raw transaction data into actionable insights for personalized marketing, merchandising and category management, supply chain and retail operations, and more.

To uncover the insights that its customers require, Symphony RetailAI must continually ingest and analyze terabytes of customer data. And in the fast-moving consumer goods (FMCG) industry that Symphony RetailAI serves, where products are sold quickly and at a relatively low cost, the company must turn those mountains of raw data into actionable insights and get those insights back into customers’ hands as rapidly as possible.

In 2018, Symphony RetailAI began looking for a data warehousing platform that could help it both handle continued customer growth and minimize the time required to give all customers the answers they need. That meant not only reducing query times, but also the time required to ingest that data, analyze it, and make the results accessible to customers.

At the time, Symphony RetailAI relied on a range of different data warehousing platforms. Specifically, the company’s data warehouse environment—comprising more than 700 TB of data—consisted of:

• 10 Netezza servers in Europe and the USA
• 15 AWS Redshift instances of various sizes
• 32 1010data servers in Europe and the USA

The problems these systems presented were manifold. AWS Redshift costs were significant and cube build processes took up to 20 hours, making it hard to increase functionality without violating the SLAs that Symphony RetailAI had with its customers. Queries on 1010data were also slow, and its custom programming language made development more difficult. Processing also took too long on Netezza, which was already at full capacity—and would soon be out of maintenance.

To cover all its bases, as the company began looking at new platforms such as IBM Sailfish and Jethro, it also worked with existing vendors to optimize performance.

Solution

That’s when Symphony RetailAI discovered Yellowbrick. The company was intrigued by its unique architecture, which promised to deliver unparalleled price-performance at massive scale. “We ran Yellowbrick through several different use cases, where it delivered three to five times better price-performance than all the other systems we evaluated,” recalls Nigel Pratt, Senior Vice President for Development at Symphony RetailAI.

Symphony RetailAI has already replaced five full-rack Netezza systems with six 6-U Yellowbrick appliances, with plans to retire another two Netezza systems before the end of 2020. The company is moving multiple Netezza workloads onto Yellowbrick, including the generation of analytics cubes, batch reporting, dynamic real-time reporting, and real-time queries from both analysts and its own applications. “Being able to easily support mixed workloads is another advantage of Yellowbrick,” says Pratt. “We’re using its built-in workload management system to ensure that each workload will always have sufficient system resources.”

The company has also converted all production 1010data systems to Yellowbrick and has moved all AWS Redshift workloads onto Yellowbrick.

Looking toward the cloud

So far, Symphony RetailAI’s use of Yellowbrick has been entirely on-premises. However, the company plans to investigate how it can take advantage of Yellowbrick’s unique hybrid-cloud architecture to run its analytics workloads wherever it makes the most sense: on-premises, in a private cloud, in the public cloud, or any combination thereof—with the same predictable price-performance.

“Being able to quickly spin-up a Yellowbrick instance in the cloud—and run it exactly the same as we do on-premises—is attractive for several reasons,” says Pratt. “For example, it could help us more quickly deploy POCs for new customers, without having to physically purchase any hardware. A cloud-based option also provides new opportunities to simplify processes and avoid moving so much data around.”

Although other solutions the company considered were cloud-based (including Snowflake, AWS Redshift, and Google BigQuery), Pratt says they didn’t offer the same predictable price-performance as Yellowbrick. For example, although one such vendor promised on-demand scalability, the company found that it would either need to subject customers to unacceptable 12-15 second delays for their real-time queries as data warehouse instances in the cloud were scaled-up, or would need to pay four times as much as Yellowbrick costs to keep those instances running 24x7.

Benefits/Results

Through its use of Yellowbrick, Symphony RetailAI is benefiting in several ways:

• Unmatched price-performance. Yellowbrick delivers 3x to 5x better price-performance than the other options Symphony RetailAI considered. It achieves this by building on modern technologies such as massive core-count CPUs and fast NVMe memory to deliver a unique, massively parallel processing (MPP) architecture that completely eliminates the traditional bottlenecks that, up until now, have constrained how much data could be “hot” at any given time.
• Faster, more accurate reports and queries When on Netezza, because of scalability limitations, Symphony RetailAI had to limit reports that were run up to 500 times each day to 10% data sets. Today, on Yellowbrick, the company is running those reports with 100% data sets and delivering them twice as fast. The migrations from AWS Redshift and 1010data onto Yellowbrick have yielded similar performance gains. Compared to AWS Redshift, cubes built on Yellowbrick are now completed 3x to 5x faster. Similarly, compared to 1010data, Yellowbrick is delivering 4x to 5x faster query speeds.
• Freedom to deploy anywhere. Yellowbrick’s compact 6-U form factor has also helped Symphony RetailAI accelerate its migration off Netezza by making data center space, power, cooling, and so-on essentially non-issues. “Currently, we have four Yellowbrick systems in Atlanta and two in London,” says Pratt. “Yellowbrick’s small form factor lets us easily run our workloads where our customers are—instead of shipping raw data and query results back-and-forth across the globe.”
• Less data movement. Strong performance under mixed workloads has helped Symphony RetailAI streamline processes and reduce data movement. “With Netezza, we had to move a copy of our data up to the cloud before customers could access it, and often ran into synchronization issues,” says Pratt. “With Yellowbrick, we can now keep all that data in one place. We’re delivering new insights to customers up to 12 hours faster each week by avoiding the need to move hundreds of gigabytes of data up to the cloud before it can be queried.”
• Ease of migration. Compared to what would have been required on other platforms, the company’s move from Netezza to Yellowbrick has been fast and easy. “With some of the other platforms we considered, migration would have required a lot of application rewrites and other changes,” says Pratt. “I can’t say that Yellowbrick was 100 percent compatible with everything we had before, but it was damned near close—requiring only very minor changes.”
• Additional technical benefits. Symphony RetailAI is benefiting in other ways, as well. For example, Yellowbrick is delivering better mean-time-between-failures (MTBF) because it has no moving parts and a design that avoids any single point of failure. Encryption at rest is also by supported by Yellowbrick, whereas it was not supported on some of the Netezza systems. Finally, Yellowbrick delivers better workload management features and a more useful and intuitive system administration console.

Looking back, Pratt is happy he chose Yellowbrick. “Yellowbrick has turned out to be a very fast, cost-effective, and reliable system,” says Pratt. Not only has it enabled us to support continued customer growth, but it has also enabled us to provide all our customers with richer insights more quickly.”