Can Machine Learning Offer Hope? Yes

Vendor Profile: Conversion Logic
by Jim Sterne

 

I always cringed when the discussion turned to attribution. First click, last click, time decay; what nonsense. The late George Box had it right: All models are wrong; some models are useful. But attribution was pure make believe.

If you happen to capture the proper inputs and happen to create a halfway decent model and happen to tweak it just so before your assumptions and data collection methods changed so much that they no longer resembled current circumstances, then it might be useful to calculate how a 30 second spot on TV coupled with a budget boost to competitive keyword bids and a flight of display ads on Facebook might result in a measurable improvement in conversions. And that model might be useful until, like all models, it suffers from the ravages of time.

My brethren, who have mastered the fine art of Uplift Modeling, are now scowling and cursing me under their breath and they are right to do so. All the tenuousness I decried above is their domain. Those who are good at it and can deliver a useful model are worth their weight in gold and are rarer than Alexandrite.

For an attribution model to secure a warm place in my heart, it would have to be a living, breathing model. It would have to be based on constantly updated, rock-solid data streams and be continuously self-correcting. Did somebody say, “AI”?

Machine Learning Attribution – No, Really

A machine that can take onboard a wide variety of data types and work out the correlations on its own offers promise. A machine that can create a predictive model based on data and not intuition – and then change its mind when it gets fresh data – well, that melts my heart and makes attribution a real possibility in my mind.

It also takes a considerable amount of data and a metric boatload of data scientists who are rarer than unobtainium.

Having read the marketing claims on the Conversion Logic website, I spoke with CEO & Co-founder Brian Baumgart, a man who has already enjoyed a healthy career as a serial entrepreneur in the online advertising and marketing technology space.

I told him I was duly impressed with their selection of algorithms for their Ensemble Model (Gradient Boosting Machine, Neural Networks, Factorization Machine, Logistic Regression, Kernel Ridge Regression, Extremely Randomized Trees, Random Forests, and K­Nearest Neighbors), and I can act like I know what each one of those is as well as the next marketer with a degree in Shakespeare.

I pressed Brian to explain what all that mean in marketing-speak. What can this robot do for me? Can it really figure out cross channel attribution? Really?

Baumgart doesn’t claim to divine the ROI of a poster on the side of a bus or the branding impact of a flying drone sphere, but Conversion Logic can tell you if your television ads are having an impact. “We have our cross-channel product which combines time-series modeling of linear media (TV) with digital user level modeling. The output is a baseline calculation of brand equity that allows us to see the incremental impact of television on search conversions, for example. We have some clients who have years of TV data.”

Next, I wanted to understand the real-life experience of the real-life client.

Turning on the Marketing Machine

Baumgart explained that sometimes, they are brought into a company by the high-level analytics group who understands the need to bring in outside technologists who have solved this particular problem before. Sometimes, a very data-driven marketing executive wants to make things happen and introduces Conversion Logic to the data people.

The technologists want to be sure it meets their high standards and is compatible with all the other systems they tend. The marketing team wants proof of concept before they can trust it with their budget. The CFO lowers the green eye-shade, crosses the arms and demands proof.

They are not ready to let the machine make decisions sua sponte. They still want to understand how the machine works and why it is making the recommendations it does.

“In the last couple of years,” says Baumgart, “we’ve been talking to more, larger companies that have the data scientists who understand our methodology and insist that vendors stop talking about black boxes. We have to be able to explain what goes on inside and when we have data scientists in the room, they like what they see. When we don’t, there’s a fair amount of ground we have to cover to bring people up to speed. And that sort of disclosure opens up the conversation about their data and their goals which is an absolute necessity. This stuff can’t just be plugged in and turned on.

“When it comes to turning over the actual execution to the machine, the technology is ready. We have API’s that work, we have done the testing and have use cases where the model is updated by the output of the campaign and triggers the next action. Marketers, however, are not pushing in that direction yet. They still want to understand the narrative and have the conversation about why the machine is making certain recommendations. That’s coming, but marketers still want to fully understand how it works first.”

Data at the Heart

Machine Learning runs on data. A lot of data. It does nothing (yet) to help normalize and munge that data between the silos in the martech stack, the adtech stack, the various CRM systems a large company may have deployed, not to mention third-party data that starting to show up. (The purchase of Weather.com by IBM is a clue .) “While ML is most often downstream from normalizing and munging,” Baumgart said, we’ve invested heavily in tools and technology to automate this part of the process, to ensure clean data flows to our Ensemble ML framework.”

Baumgart explains that, “the onboarding process begins with a frank conversation about which data is the most likely to be useful, taking accessibility into account. Even a small company is spending on Facebook, search, affiliate marketing, and they have some CRM. That’s already four data silos to bring together along with some blind spots. We need to do some meaningful discovery, a process that is absolutely necessary to getting things started.

“There is a good deal of data mapping and marrying and massaging that needs to be done and we’ve built a lot of capability into our ETL and into the foundation of our platform to help automate that. We also have a whole team on the solutions architecture side that is looking hard at alerts and sanity checks and tests the data and QA’s the data. We’ve automated the process of looking for anomalies, but it’s not hands-off. Humans are still needed to understand nuances by channel or by tactic and we train the machine as best we can to learn the difference between normal and abnormal variances.”

Domain Knowledge is Table Stakes

This is where experience is helpful. Conversion Logic has already done lots of integrations with call center providers and has collected conversion data from websites, mobile apps, and in-store point-of-sale systems. The hard part is bringing that data together.

Baumgart describes the ideal, day-one client as being attribution-ready. “They have the tools and processes that make up a strong data governance methodology and that means they have consistent low-level policies around campaign naming conventions. That gives them data integrity and access to data through the right tools.

“We walk each client though our Attribution Readiness Assessment which is a collaborative exercise with the brand that goes through a list of best practices but ends up with us understanding what people, processes and technologies they have in place. It lets us identify holes and gaps. Some of those things are easy to rectify, and we can help them where it takes a little more time. Otherwise, we’ll go through solutioneering, where we’ll identify which things will have to wait until phase two or three. We want to clearly spell out the path to value.” (See: 3 Ways to Validate Your Attribution Data, Conversion Logic blog post.)

Secret Sauce: Path to Value

Finding the data is a start, and then there’s the plumbing which has to be rock solid – getting the proper data, in the right format, to the right location for processing – the operationalization side of the equation. Then one can train the machine to derive a useful model.

How, I asked Baumgart, does Conversion Logic actually stitch together behavior between channels? “We’ve put our energy into and make some big innovations in maintaining the continuity of user identity across devices for the highest possible level of pathing coverage. This feeds into the benefit of an ensemble machine learning framework instead of depending on a single algorithm. We’ve identified the best algorithms and methodologies for each piece of the puzzle and then mastered a way to tie them back together.

“For many clients, this is an opportunity to take a holistic view of their data by talking about their KPI hierarchy, their P&L structure, the org chart and the decision-making process. This really helps us set the proper level of expectations for various stakeholders and can identify some practices that they may want to change. That can sometimes be threatening so, we really focus on getting incremental value as soon as possible,” Baumgart asserts. “We want everybody to understand that this is not a silver bullet.

“Once we start training the machine, we can come back with optimization advice on TV alone before we even throw the digital data in there. Within a few weeks, we can point out two or three opportunities to optimize TV spend. this is the incremental path-to-value that we go after to avoid that problem of waiting months and months before anything comes out of the system. We want to get to value within 90 days. That brings us into the conversation about KPI’s. You have to pick what you want to optimize and that is a business decision that is sometimes political.”

Jim Becomes a Convert

Having written a book on using artificial intelligence in marketing, I now know enough to pierce the jargon and see the potential. Some companies, like Conversion Logic are bringing the vision into focus and helping firms like GM, HP, and Pizza Hut get closer to their customers than ever before.

I don’t cringe at the mention of “attribution” anymore.