The Usual

The first few weeks were very hectic for me. Jeffrey Klein and Julian Brookes, two of the senior management team members were trying to analyze the results of a survey conducted on a national group and they needed me every moment to write program, create charts, tabulate results or whatever came to their heads. They were obviously strategic thinkers. If they gave me some time to analyze the data by myself, perhaps I could have come with some valid conclusions but trusting such a work to a junior person would not have been proper. So they struggled, asking me to do things only brilliant people like them think of, and as is usual with brilliant people, they failed brilliantly. A good thing that came out of this was that my interactions with these two gentlemen slowed down which meant less of those finish-this-2-days-work-in-1-hour assignments.

We had a trial customer, a customer whom I will call CU. Our contact John was gullible enough to believe in what Jeffrey Klein and Sumer Johal told him, and they gave us some money, opened up their customer databases, and allowed us to test out our models. I heard that he was upset because we promised them that our sophisticated algorithm designed by world's leading experts will have different questions for different customers but because of the glitch in our software, all of them got the same 10 questions. All 10,000 customer who took the online survey. Wait till he finds out how Xamplify took him for a complete sucker. Then he will be really, really upset, I guess. (Sorry John, xamplifysucks.com or xamplifysucks.org domain names are already taken!)

We conducted the online-survey on 10,000 of their customers. (This number as well as other numbers used here have been altered slightly.) We claimed that based on their responses, we knew their psychological profile consisting of 10 psychological traits like price-sensitiveness, financial-savviness and so on. (Response to one question revealed their one psychological trait. There were 10 questions.) Now we had to extrapolate those psychological profiles to all the 500,000 customers. How did we do it?

Simple. We had demographic (e.g. age, sex), transactional (e.g. time of last ATM usage), and product ownership (e.g. platinum visa) data available for all of them. All 500,000 of them and not just for the 10,000 who took the online survey. We used that data to link as follows: For 10,000 customers who took the test, for each question (equivalent to one psychological trait) we fitted a linear regression line which used the other data to predict the psychological trait. In linear regression techniques the underlying assumption is that the relationship is linear. My preliminary work and the common intuition said that there were many problems with the assumption of linearity but who was I to influence the important decisions made at Xamplify?

Let's say we got an equation:

price.sensitive = 0.8 + 0.01 * Age - 0.03 * account.balance

There are various ways to check whether the models we got makes any sense or resonably reflects reality. One is error diagnostics where we check how well the model fits the actual data. We didn't check that. There is a number called R-square which can be from 0 to 1. 1 is ideal and indicates that the model explains all the changes in the dependent variable (price.sensitive above) and 0 is the worst which means the model didn't explain a thing. In our case starting R-square values of 0.02-0.03 were common which were later tweaked to slightly higher values of 0.05-0.07 which still showed weak relationships and meant that these regression lines were not that trustworthy but who cares. Right?

Now we have ten equations, one for each psychological trait, and all derived from 10,000 customer data. Now using the same equations, by plugging in the values of the variables on the right-hand-side of the equations (e.g. Age and account.balance) we got the dependent variable values (price.sensitive in above case) for all 500,000 customers. How good these numbers were? In company's view "accurate." How would you feel if CIA starts torturing you because you frequently sent money to your mom and these equations told the CIA that you are a risk-taker and there is a moderate probability of you being a money launderer? Claims backed up by Xamplify's top-tier experts!