Simon Baker: So come up and join me here. We’re going to have a real deep conversation, I hope, around how sort of AI machine learning can be sort of really practically applied. But let’s first things first, introductions. So Roman, introduce yourself.
Roman Sidorchuk: Thank you very much for the invite. Really looking forward to this conversation. I am a machine learning engineer at a company called Metropolis. It’s a computer vision startup for parking. I do dynamic pricing. In the past, I was working at eBay, Meta, NASPERS, Avito, and McKinsey.
Ted Rubin: OK. I’m Ted Rubin. I’m the CEO and executive chairman of ActivEngage. I have been in the automotive industry for about 30 years. I used to run dealerships. And then I started a CRM company. And ultimately now, we have a messaging company, which is live messaging, which handles the messaging for companies like AutoNation and Cox Automotive and a lot of others, General Motors.
Simon Baker: Fantastic. Well, let’s get into the exact topic around sort of AI, right? Because what I’m curious about is how do you see it fundamentally changing the way in which your part of the world operates? So you’re dealing in pricing dynamics. How is AI changing that outcome?
Roman Sidorchuk: A few things. I also worked in pricing in my previous roles at eBay and NASPERS. What I do see is that clearly there is more technology and more use cases for pricing in machine learning. And also, technologies are getting more sophisticated. And people are getting more experienced with technology. For example, I just started onboarding with one of my coworkers in product management. And they are actively using Python and SQL themselves in the past.
Roman Sidorchuk: They clearly would ask an analyst or a data scientist to have this type of technology. A lot of new developments coming up, of course, with different research papers and articles. So my team, we clearly do keep up every week reading different research papers. So those are, at the same time, the third trend that I think is still important to know is that, by far, the biggest challenge in applying machine learning projects is not technology or coding. This is always still, in my experience, organizational and business topics.
Simon Baker: Happy to talk about that more. Let’s dive into that a bit. So you’re talking that the bigger challenge is in organization. What do you mean by that?
Roman Sidorchuk: A few things. For example, I have one example of our recent project. Project lasted six months. And I asked my teammates how much time was, what type of work, how much time it took. Actual coding was one day for in six months project. A lot more time was about different aligning different systems, talking to different teams, getting approvals, all this, having spread out data, have it aligned and collected. So all this preparation work and alignment takes a lot more time than the coding itself.
Simon Baker: So the way you view this is that AI is a great concept. And applying it to pricing and such is critical. And we’ve seen from a number of these guys previously where the dynamic pricing that’s being done, for example, in the UAE with the property finder, et cetera, to extract more value or even in auctions. But from your perspective, the coding’s easy. It’s actually getting the organization aligned around the new business model.
Roman Sidorchuk: Exactly.
Simon Baker: And how did you approach that problem? How did you start to get everyone aligned around, we’re going to change the way in which we do pricing. We’ve got to communicate it to our customer base. What were some of the mechanisms used to get alignment and thus get a good outcome?
Roman Sidorchuk: For me personally, this is relatively easy because I come myself from business background, from management consultant at McKinsey, where you need to be very proactive and learn very quickly and learn with different stakeholders and types of stakeholders, both technical and non-technical, from IC, all the level up to board of directors. At the same, so I ensure that I build relationships and trust with stakeholders.
Roman Sidorchuk: I get up to speed and know what is their top of mind and build those productive relationships and bring value. At the same time, usually there is a big gap between, like, let’s say, technical people. Like right now, for example, my team is engineers. So very big, they are very advanced technical skills. But when it comes to stakeholder cooperation and understanding the business impact, usually there is a big gap.
Simon Baker: OK, I can understand that challenge. Having both worked in management consulting at McKinsey and have also been in technology myself and seen that that gap and bridging that gap in a way that leads to a positive outcome is always very, very challenging. Ted, let’s talk about ActivEngage and what you guys are doing there. How do you… Well, first of all, and we had a bit of a chat last night, how do you see AI changing what you guys do?
Ted Rubin: Well, AI is a great tool to be able to help guide the direction of the conversation, as well as to be able to guide the agents as far as where to go and sort of predict what to process with the customer and what to suggest for the customer. So we’ve always looked back at the historical conversations and we have 17 years of these conversations. And so we try to mine that information historically to be able to tell where and how to train our people to correspond and to interact with the customers.
Ted Rubin: So as you deploy AI for that, it can do it much faster. So, and you can assimilate a lot more information. Some of the difficulty in using it is that, especially as the evolution of the environment of people buying cars has sped up a bit and their perception and their utilization of these tools has changed. Some of the older information is not as relevant to the current conversations that it is today. So you have to kind of monitor what’s actually being put into these models so that it’s the more current information. But what’s great about AI is that you can do it very, very quickly with very current information and have instantaneous output.
Ted Rubin: What we don’t like, what we were talking about the other day is that I don’t like having this, I like having it as a tool. I like having it where it’s assigning and prescribing information for the agent. What we find is a failure is when it’s on the front end trying to communicate directly with the customer. So at least at this moment in time, the way that it gets utilized, the customers are interacting with this particular conversation and when they click to chat or if they’re interacting through any messaging, even if they click the call, their expectation is to speak to a live person mostly because they want the advocate, they want the interaction, they’re looking for some empathy, they’re looking for some ability to go, hey, you’re doing a good job, this is a nice vehicle or this is a nice product.
Ted Rubin: And the machines don’t do that very well. They can do it okay, they keep getting better and better but usually the way they’re deployed if they’re on the front end, they wait till failure and that’s a problem and that’s what we were talking about last night. Now you have a pissed off customer and the agent that’s gonna take it live afterwards doesn’t wanna take it because they’re already upset. So we like to look at using this sort of intro to the conversation. So more as a tool, more as research, more as pulling the data out so that during the conversation it can make suggestions to the agent and that way the agent can choose what it wants to promote or if it wants to enhance or change what it wants to promote and that keeps evolving as well.
Ted Rubin: Plus it feeds the model much better so the machine learning is getting what it’s promoting and then you’re saying, okay, I’m gonna use it as it is or I’m gonna embellish it and then all that gets fed back into the machine so you’re getting more quantifiable data when you’re feeding the machine learning system so it can put out better information later. We also use it on the very back end of it to be able to just look at the breadth of everything that’s going on and look for anomalies.
It’s hard to find, you have to kind of suggest or we used to have to say this is what we’re looking for. Is there anything in this line? Is there anything in this line? With AI you don’t really have to do that. You can just go find me anything that looks like an anomaly as small as it may be. If there’s phenomenal consistency, then I wanna see that and let’s see if there’s anything we can do conversationally to tackle that in a better way or address it in a more meaningful way for the customer.
Simon Baker: Can we bring up Slido and allow people to ask questions as we go? I wanna explore this a bit more because it’s really a similar sort of issue, right? The technology is not the hard bit. The hard bit is how to integrate it into the day-to-day operations of the business. And if I understand correctly with that ActivEngage, if you go back in time, it was really just providing channels between consumers and company or call centers or whatever is at the other end, right? And you would then apply some intelligence on top for call frequency and a bit of analysis around it. Now with AI, I assume does it listen in?
Ted Rubin: It can listen in, right.
Simon Baker: So it’s listening into the conversation but it’s not replacing the call center at the moment?
Ted Rubin: No, it just hasn’t gotten to that stage at this particular point. And I think also one of the innate problems to it, as we mentioned it last night, which is the volume of data, even though we’re essentially the largest provider of this service for specifically for this industry, there still isn’t enough data. You need really large quantities of data to really kind of extract really significant pieces. So what we don’t want is for it to fail at any point. So everything is being promoted to an agent who can say, yeah, I wanna take it as it is, or I wanna take it but I wanna embellish it and then it actually gets up to the customer.
Simon Baker: Okay, so because I see these stepping stones in sort of it becoming, going from way in the background to being an assistant and then potentially taking over over time. And if the stepping stone has to be going from handling the first part of the conversation, because these chat bots where you’re asking relatively benign questions and it comes back and answer and eventually says, oh, I’ll put you through to someone who can answer that. I’m gonna put you to a real person who’s dynamic enough. To if it’s now listening in the conversation, are you using then AI to prompt the call center person or whoever’s on the other end on what to do next?
Ted Rubin: Yeah, that’s the way we’ll use it exactly. We don’t want it on the front end, initially facing the customer because that’s a relationship issue. So that’s building trust, which we’ve talked about throughout this entire conference and that’s building an interpersonal relationship which is motivational for the customer to wanna continue to stay engaged, to buy from that particular vendor that we’re representing. So that needs to be built with people at the moment. These machines just aren’t that great. They’re phenomenal at regurgitating data, giving you definitions, identifying things really quickly, customers ask detailed questions.
Ted Rubin: But you have to be further into the conversation for that. So rather than putting it on the very front end, we have it monitor and then make suggestions to the agent in the middle.
And so someone might be, what do you think about this car or I’m interested to get this. One of the big things that we ask is why and we have to kind of ingest that and the representative has to understand what that is. But then as you get more into it and they say, well, how late are you open or how much is this car? Is there a way for me to find out if I can buy it through financing and stuff? Well, some of those questions, the AI, it’s just quid pro quo, right?
Ted Rubin: So it’s easy to answer and the AI can look at it and go, oh, well, here’s the answer to that question and here’s the answer. So rather than having X amount of conversations per rep, I can have one and a half or two times that volume per rep and they can still give their specific attention to it. They just don’t have to type out the responses and they don’t have to think about what the responses are. They can read it and go, that’s it or no. Or they can enhance what’s there and it’s much quicker and they can handle more volume at the same time and that’s the cost efficiency of it.
Simon Baker: So I see there’s sort of two issues here. One is, today it’s making the end customer care personal, whoever, far more efficient and effective in two dimensions. One is they’re not wasting time having to regurgitate the basics, right? Because it’s just fundamentally, it’s… It’s static. Yes, the car is black, right? It’s got 10,000 miles on the clock, got it, right? The other thing is that it’s making them more effective for the organisation because it’s saying, by the way, don’t forget to ask about finance. By the way, don’t forget, ask this question or that question because you don’t know if they’re qualified, right?
Ted Rubin: Correct, it helps with the process. Although I think from a training standpoint, I think that we’ve been able to, over time, we were able to train that specific piece, the training process, pretty well. So there is kind of a methodology that they go through and they can do it. So it’s helpful early stage for new representatives to be able to watch that, I think.
Ted Rubin: I think where the more significant piece to us is the quantity. It’s just volume of communication per representative because we’re very focused on the quality of interaction with the customer because that’s ultimately what produces not just more leads for our customers, but usable ones. Ones that where the customer’s more… Anybody will give you their name today.
Ted Rubin: We were talking about PII the other day. We were talking about gathering personal information. And if you ask someone what their name and phone number is, then they’ll give it to you. But will they take the phone call is the question. So there has to be this relationship building that makes the customer feel like, okay, this vendor has given me what I’m asking for. I feel like, okay, I wanna have some, I’m gonna go further. And so when I’m gonna take their call, I’m gonna look at their vehicle, I’m gonna do these other things.
Ted Rubin: So that still needs to be built, it seems like at this particular time. And as the AI evolves, as it becomes further down the road, as we get to a level three or self-awareness of it where it can actually have emotion in the conversation, then obviously it’s gonna be able to have much more effectiveness. But at this moment, where we’re seeing it’s utilized most effectively is in making the suggestions and then as well, just shortening the focus and the intensity of the conversation for our agents so that they can be freer to handle more conversations at the same time with the same level of quality that they would if they just had to read through everything.
Simon Baker: Makes sense. Now, Roman, I’ve got a, there’s a question come up which I think is quite interesting. If organizational problems hold up these machine learning projects, right? Because you’ve got, you said one day of coding out of about three or four months and the rest is all about within the organization, then the question is can AI help exactly solve that problem?
Roman Sidorchuk: Sure, a very interesting question.
Simon Baker: And if so, how do you see that occurring?
Roman Sidorchuk: In some cases, yes, and others, no. So case where it’s applicable is, for example, let’s imagine you have like different teams and a lot of documentation, AI technologies, especially generative AI, large language models such as ChudGPT or CloudAbiantropic, they do can understand and combine and summarize a lot of information. At the same time, there is a challenge, of course, with privacy because I think one of the questions in the previous sessions was whether you can apply ChatGPT, for example, for company proprietary information.
Roman Sidorchuk: In some cases, it’s clearly not allowed so that company needs to adapt the policy and to ensure that this information stays confidential. Another example is that a big problem for ML projects is that you need to have enough quality data and AI can actually help to generate sometimes synthetic data or get the data from third partners with parsing.
Roman Sidorchuk: And a third example is that I think AI can also help when you have limited resources, which can be another problem is that it can also help employees be more productive. And do more with fewer sources. So those are some examples where AI can help. At the same time, there are, I think, more fundamental organizational problems where AI cannot help.
Roman Sidorchuk: One example is that you can, back from my even old days in management consulting or investment banking, you clearly can have situations where you just can have teams with competing priorities and it’s a people problem, it’s not AI problem, right? So if you have one department wants one thing, another one wants another thing. Or if you have a problem where, I think it was also, the previous speaker also mentioned on the dynamic pricing in classifieds, but it’s also applicable for even our business is that you might have different optimization goals.
Roman Sidorchuk: So for example, in one case, you might want executives and investors might want to get, let’s say more impact in terms of dollar revenue amount. At the same time, this might not always be sometimes conflicting with the goal of getting more transactions or making customers happy. And another example can be also flow information between the companies. So there might be one, on executive level, there could be one example and one goals. At the same time, on a team level, especially for technical teams, it can be very different. So those cases, I think it’s more about the culture, organizational culture and information flow rather than technology.
Simon Baker: What’s interesting is that if you think about how organizations are structured and how OKRs, et cetera, push down through organizations, I would have thought that’s a pretty good application of AI where you feed in, here are our organization goals.
Simon Baker: We need to grow revenue by 30%, we need to, whatever. And out of that, you can pretty much generate because if you think about it from your consultant days, you’d probably remember this sort of stuff. Everything’s sort of procedurally pushed out. That’s a way which I think the leadership team within businesses can start to use AI internally to start to get consistency and then of course the AI can sit there and monitor the performance and report back seamlessly without having to have human necessary interaction around that. So then we have the machine setting the goals and managing us, which is a scary thought, right?
Ted Rubin: But it’s a practical thought though. It’s practical relative to, you’re almost describing some of the things that we look at for the same thing because it is this oversight piece. We’ve historically had team leaders who will watch multiple conversations at the time and interject if they feel like there’s something that’s necessary and help manage the people who are actually directly communicating and using the AI,
Ted Rubin: You can have that do that for all of the conversations. I don’t need six or seven or one per eight people or whatever it is, I just have one AI piece that can do it for the entire company and watch all the conversations and then continually use that information to determine intention for the customers over a longer period of time. I don’t think we’re quite there yet, but as we can get to that, then it’ll start to kind of assess where do we think these customers are going?
Ted Rubin: Some of the last speakers were talking about that and if I think they’re going in this direction, then we wanna promote these responses. I think that exactly to the point that you guys are making right now, that it’s being used that way.
Simon Baker: Yes, and eventually we will work for the machine, so we’ve gotta get used to it at some point. Is it more effective to fully replace the call center or operate with AI if the communication between the buyer and the call center is conducted through WhatsApp?
Ted Rubin: There’s still – I don’t think so, not at this time. It’s the same issue of intention from the customer. What are they trying to accomplish? 81% of the customers that click to engage for any purpose, if they’re initiating the conversation, then they wanna talk to a person. They actually, that’s the answer that they’ll get. They want to talk to a person. It’s not they’re trying to find research or they couldn’t find what they wanted on the site or they have a question that specifically needs to be answered,
Ted Rubin: That’s not the answer they’ll give you when they asked, why did you click on messaging? It’s that they wanna talk to somebody. So you have to give them what they’re actually asking for. So I don’t think the medium matters. I don’t even think that if you call in, then I don’t think it matters they don’t wanna talk to a computer right away. That doesn’t mean that that can’t be interjected at some point. And I think that it’s gonna continue to evolve as the AI gets better and it gets more fluid and it has more to draw on so that there’s not the mistakes that it does make even if it’s only making 20% of the mistakes,
Ted Rubin: 20% of the time that’s not the whole amount and that’s about where it is. And it doesn’t, what we’ve seen at least conversationally it doesn’t just fail, it fails catastrophically. It’s what color is the car and it says a building or something significantly wrong.
Simon Baker: And- And that’s just a function of time, right?
Ted Rubin: It’s just a function of time. Function of time. Oh yeah, absolutely. And becoming more. There’s no question that the AI is gonna be able to handle the vast majority of it. It’s my personal opinion that there’s always going to be some requirement for human interaction. People wanna touch stuff and they wanna feel like they’re connected to some type of humanity. I don’t know that that’s gonna be, I think it’s gonna be pervasive but limited in quantity massively as the computers get, as the systems get much and much smarter and much more capable of handling it and handling it more quickly.
Ted Rubin: I think as we get more personalized and the computers can narrow the scope and the time that you have to be involved down so significantly then it’s gonna be an appeal actually to the general consumer. They also have to get acclimated to it. You know, we got acclimated to Google and now anything that takes longer than two seconds seems like it’s taken forever, you know, as a search.
Ted Rubin: So I think that you’re not gonna have to wait for them to be perfect. You’re gonna have to wait for them to get better and you have to wait for customers, consumers to go, yeah, I’m okay with this correspondence. And when those two things meet then the AI will take over much more massively in the conversation. It’ll be 90%, 95%, maybe 99%. But, you know, hopefully there’ll always be some limited use for people.
Simon Baker: They’re probably to collect the dividends. Okay, a question on the engineering side. This is sort of one that I’m fascinated by, right? Is how redundant will engineers become with AI? Because AI can start generating code, it works 24 seven, doesn’t need to be fed pizza and doesn’t have attitude. So how, or maybe it does, we’re not sure yet. How do you see the role of AI in actual the engineering side of a business?
Roman Sidorchuk: AI clearly is helping engineers to be a lot more productive. So I would say clearly you can have one person do substantially more work and that’s happening already. And in the future, of course, there’s gonna be more and more of that. I don’t think when it comes to production level code and especially like actual like implementation deployment of the systems, not gonna be like short term replacement.
Simon Baker: What about medium term?
Roman Sidorchuk: *laughs* Possible.
Simon Baker: Long term?
Roman Sidorchuk: Definitely. But I also have a son. So my son should study coding. But I have one interesting actually example from the previous question, from like two little histories, like one for like call center, let’s say you’re a classified website and then you have like biggest like, let’s say customers, automotive dealers. For example, in this case, probably you don’t want to have like AI system. You clearly need to have like personalized human touch and the person who can like reply quickly because this is like very different case compared to let’s say when I was working at Meta. One of things I repeatedly got complaints from my friends who got their Instagram accounts blocked for various reasons and they didn’t have account support before they launched Meta Verified.
Roman Sidorchuk: So for billions of customers, right? They cannot have human call center because it would be uneconomical. So that’s why having automated AI service even to some extent would be for sure beneficial.
Simon Baker: I agree having had my account blocked on Facebook for ads. No idea why they haven’t told me yet that is blocked and they say it’s blocked. So how is AI gonna change data science and analytics within organizations? So Roman, how do you think that will change?
Roman Sidorchuk: A few things first is that clearly got more technical. For example, when I was just starting out my career like NASPAR Savito in tech like six years ago, it was you could just say that like you know like SQLs maybe some basic tableau reporting experimentation.
Roman Sidorchuk: Nowadays, at least in the US for senior roles, there was you clearly data scientist is the one, expectation is the one who know like system design, who know how to code, who know how to deploy systems end to end so it’s more closer to like machine learning. So there’s a – I would say this is the biggest trend that is getting a lot more competitive and a lot more technical.
Simon Baker: Okay, excellent. We’re gonna finish there. So thank you very much, Ted, Roman for sharing your thoughts around this. It’s sort of refreshing to feel that well, it’s here, AI is here to assist us in the short term, okay? Help the coders, you know, generate more code, be more efficient, more effective, do the, as we talked yesterday, do about the quality control and so on, et cetera. It’s good to see that it’s being applied in your business around assisting to get better outcomes, assisting people handling at the other end of that chat.
Simon Baker: But it does raise the question of what the future does look like. And it does raise the question of what does the organization of the future look like? Not just the marketplace, but any organization of the future look like if code is being generated by AI, call centers being handled by AI, the finance team will probably be AI driven, right?
Simon Baker: What’s the role of people gonna be in this? Not to be answered right now, because luckily I’m old enough not to have to worry about it. But if the kids growing up nowadays, you’re gonna have to really start to scratch your head about what’s the role that you wanna have in this outcome. It’s inevitable, there is a degree of inevitability of some of this occurring. We just don’t know if it’s going to be a massively cooperative society or it’s gonna be something like out of us, was it Kubrick’s where they open the hot bay door hall and he’s gone, no, I’m not doing that.
Simon Baker: So thank you very much for sharing your thoughts and ideas around this. And I look forward to seeing you for the rest of the conference. So thank you. Thank you!
Roman Sidorchuk: Thank you!
Ted Rubin: Thank you!
