YRGLM

Solution

Column database analysis processing

OpenText™ Vertica™ Analytics Platform has been introduced to 3,500 users worldwide as a column database. Its main feature is its outstanding analysis performance, which is typical of column products. The generic, line oriented, relational database has to read all the subject data during searching, whereas Vertica Analytics Platform pinpoints and reads only the necessary columns. Wasteful processing is eliminated, keeping disk I/O to a minimum, which realizes high-efficiency, high-speed analysis performance.

“With Audience EBiS customer journey analysis, customers use a free approach to perform analysis requests on 10 billion pieces of measured data. For example, when a video advertisement is included in the marketing measures, it is possible to clearly show how the conversion rate (CVR) or purchase unit price changed and what the comparison would be if video advertising was not done,” says Uchida.

As with the video advertisement mentioned here, with the initial stage measures for potential segment marketing, the return on investment is difficult to see. When the measures are evaluated simply by the cost per acquisition (CPA), there is a risk of missing a business opportunity.

“Vertica (now OpenText) almost instantly visualizes analysis results for atypical processing requests such as how to change the analysis axis or how to change the tabulation unit and do the evaluation. This is performance that can’t be realized with a generic relational database.”

With a generic relational database, fixed performance improvement can be achieved only when there is routine processing using an index. This is no match for the ultra-high speed processing of Vertica Analytics Platform. When the subject of analysis is a huge flow such as with customer journey analysis, it was not possible to put this into service because results could not be obtained within a realistic time.

Uchida explains: “To make a selection, we presented our performance requirements and investigated multiple technologies and products as candidates. We investigated not only on-premises products but also the cloud. Being able to introduce Vertica (now OpenText) as a software product, and being able to use typical SQL text and to expand processing functions and scale as necessary were decisive factors.”

Using Massively Parallel Processing (MPP) that does not have a master node, Vertica Analytics Platform can expand to linear performance simply by connecting x86 servers in parallel. Column type + distributed processing + scale out architecture is a feature of Vertica Analytics Platform. Also, using a licensing system that focuses on data capacity, there is the advantage of a stable licensing cost even if the number of server units is increased to enhance performance.

Uchida and his staff implemented performance verification using an actual Vertica Analytics Platform device. First, about eight billion pieces of measured data were stored in a three node structure system, SQL was executed using Audience EBiS, and the response was measured.

“The minimal configuration of three units gave adequate performance. However, it was not able to satisfy demands for high performance exhibited under special circumstances. We scaled to five nodes and 10 nodes and carefully evaluated the extent to which processing performance could be expanded. The results greatly exceeded our expectations,” says Uchida.

It was verified that, with the Vertica Analytics Platform, approximately double the performance was achieved with the five node configuration, and three and a half times the performance with the 10 node configuration. In fact, with the Vertica Analytics Platform, node addition is possible without stopping. When adding nodes, while the user is executing a query, data copying is performed in the background and when this is completed, it is possible to reference areas including the added nodes. With verification, a procedure for adding nodes that does not affect service and operation during failover were also confirmed.

“Enhancing system processing capacity without having to stop the Audience EBiS service is a really key point,” says Uchida. “With column databases, there are also products provided in the appliance format, but we found out that stopping is required during system enhancement and that there are scale out limits. The only product that satisfied all our requirements was Vertica (now OpenText).”

Typical SQL text can be used with Vertica Analytics Platform so tests proceeded smoothly. Verification with actual machines ended after about two weeks and with Audience EBiS, both the Customer Profile Function and the Customer Journey Analysis Function were released in 2016.

Results

Further evolution with machine learning

The Vertica Analytics Platform production environment that provides service for Audience EBiS started from a system of five physical servers and data capacity of 11 TB. Uchida comments: “Vertica (now OpenText) exhibits excellent performance even standalone without any fine tuning, but to add even higher performance, we use Projection. Immediately after loading data in Vertica (now OpenText), data placement and sorting are optimized automatically for each distributed node.”

Projection is a function that automatically performs optimization to match data characteristics and query trends. The optimized content can be edited freely so it is possible to do accelerated tuning specializing on important queries.

Also, Pattern Matching was important for customer journey analysis. Via exposure to what kind of information on which site, after clicking on which banner was the purchase point reached? With a certain behavior history as one pattern, it is possible to instantly extract only measured data that matches this.”

From measure-based analysis to people-based analysis, Audience EBiS indicated new guidelines for measuring the effect of advertising and future challenges are already being worked upon. YRGLM defines the image it wants to achieve as a Marketing Robot Company and is pushing a strategy to realize that vision.

“I’d like to work on Machine Learning using Vertica’s (now OpenText) prediction function. With the Audience EBiS customer profile, it is possible to handle information such as age group, gender, area of residence and interests. By using Machine Learning, even if this kind of information is not collected, it is possible to make predictions from behavior. It takes on the role of a concierge or an advisor,” says Uchida.

AD EBiS not only measures advertising effects, but is also equipped with a function for recommending which measure has the highest return on investment. This is an important step toward realization of a Marketing Robot.

Uchida concludes: “On an analysis platform using Vertica (now OpenText), we believe we will be executing more complex tabulation processing and Vertica (now OpenText) is perfect as an environment for repeat learning at high frequency. The world is already almost at a point where suitable advice will be presented considering analysis results and measures with the highest return on investment will be executed automatically.”