I’m so used to hearing about high profile marketing automation solutions crunching profile data to create highly targeted audience segments, that I was a little shocked to hear that they don’t do it fast enough, or at a large enough scale.
But that was the message I received from Splice Machine VP of Marketing and Product Management Rich Reimer when I spoke with him at Oracle Openworld last week. “She’s in the website, in your store, on social media, The window to reach her is the last five clicks–that’s at least near real time. The problem is, with traditional analytics you need to ETL (extract, transform and load) the data.” And with huge data loads that can leave you a day behind.
“Marketo is good for hundreds of thousands” of data points, he said; “Unica and RedPoint for tens of millions. But when you pull a hundred million people, and slice and dice them to get down to your audience, these are intense queries.” And then there are the competing OLAP and OLTP lanes. “How do you handle these mixed workloads for digital marketing?”
Well, first up, don’t panic. After all, as Reimer admitted, running queries against hundreds of millions of customer touchpoints isn’t a concern for small to mid-sized B2C operations–and not a concern for B2B at all; the waters where major marketing automation suites are still fishing. But a B2C operation with millions of customers conducting hundreds of digital transactions–from clicks and opens to purchases–may need some help.
Don’t panic also about acronyms like OLAP and OLTP. That’s insider jargon for analytics queries as opposed to transactional information. It’s like slow trucks and sports cars, as Reimer explains it: you don’t want them both in the same lane unless you welcome traffic jams.
Splice Machine can power unified customer profiles for DMPs (putting “a DMP on steroids” as Reimer described it). It’s capable of driving analytical operations across very large volumes of data (and customer profiling for marketing purposes is only one of its use cases). First, it uses a scale-out architecture to store and process data. “Scale-out” means that both data and processing power are distributed across multiple nodes in the system: “scale-up,” essentially, means adding storage space to a system which remains centrally powered and controlled.
Second, it uses a Hadoop RDBMS to manage and query the data. Again, don’t panic: Apache Hadoop is an open source framework which allows high speed access to data distributed across many storage containers. In other words, it lets you keep your data in many different places, but dip into them all at once–and fast–when you need answers.
Third–and here comes the secret sauce–Splice Machine adds a metadata layer over Hadoop to make it faster and less cumbersome, and solve the problem of directing fast transactional traffic, and slower-moving analytical queries, into separate lanes. Specifically, it overcomes the need for repeated manual rewrites of applications as the volume of queries grows. The insider take: it matches HBase storage capacity with Apache Derby SQL database analytical capabilities.
So where does that leave us? It means, for example, that Harte Hanks, a long-established marketing services company with roots in direct mail, found a new way to mine insights from the multiple terabytes of data accrued from running campaigns for major retail brands. Replacing Oracle RAC databases with Splice Machine RDBMS, they saw query speeds increase three to seven times at a fraction of the cost. A 180 second query on Oracle RAC–and remember, we’re talking about huge numbers of queries–came down to 26 seconds. “It was driving them nuts,” Reimer told me, “and creating all kinds of performance problems.”
To be clear, Splice Machine isn’t interested in storing this data. Customers can store it in their own space or in the cloud (using AWS for example). It’s a solution, said Reimer, likely to be of interest to business handling more than a terabyte of data. The solution is marketed directly to brands, including big banks and credit card processors as well as consumer digital marketing services. The need for analytical power on this scale is hardly universal, but to the extent the big marketing clouds are adding B2C to their B2B capabilities, the challenge of managing massive quantities of consumer data can’t be ducked.
“We’re talking about unprecedented scale,” said Reimer, “executing massive campaigns with real-time views to make decisions in the moment.”
