Last week marked the inaugural HumanX conference, a convening of leaders, technologists, policy makers, and media, all brought together to discuss the state of AI and its potential impact on the future of software, business, and society. Christine Yen, Honeycomb CEO and Co-founder, joined Bin Ni, Head of AI Developer Assistant at Google, Brian Goffman, Senior Advisor on AI at McKinsey, Jason Johnson, Co-founder of Doma Home, and Darrell Etherington, Founding Editor at The Angle, to explore the future of software infrastructure in the age of AI.
Here are the key takeaways from their conversation, lightly edited for readability. View the full session here:
Tackling the unknown unknowns of AI in the enterprise
“When the enterprise adopts AI technology, usually there’s one big interesting concern: what are the consequences? What will cause problems? How do you manage or mitigate the risks?” posed Bin. “When we try to deploy technology inside Google or somewhere else, we first try to create a measurement system to observe what’s going on there—to have the automation to say, hey, when you’re deploying this AI system into the picture, how do I know if it works better or worse?”
This is why the ability to observe your systems is so important when any AI tech gets deployed, according to Christine: “Measurement is so critical for closing the feedback loop in the face of rapid iteration. A lot of these AI features that we’re releasing or incorporating into products, or whole AI products that we’re launching, are really challenging to ensure correctness in a vacuum. They have to be tested. They have to be put out there in front of real users. And these measurements have to be captured in real time, accessible as quickly as possible for folks to be able to make the necessary adjustments—whether you’re at Google scale or not.”
“That’s one piece of advice I would give everybody,” Bin asserted. “If you haven’t already, please start thinking about how you evaluate, how you measure the quality or the result of your AI implementation. Once you know that, the good news is at least you know what’s going on.”
A new era of data creation and control
With the emergence of open source models like DeepSeek, the rapid improvements to models from OpenAI and Anthropic, and the investment in homegrown, proprietary models, enterprise companies are faced with a new set of options for how they manage their data and the value it adds to their specific business.
“Certainly my bias and my hope is that companies should pick the options that are most appropriate to their needs,” said Christine. “Some companies will have extreme privacy needs and require different things from their models. Other companies, maybe the functionality they’re pointing AI at isn’t quite as sensitive. And they find they can move a lot faster, a lot more economically by using an off-the-shelf model.”
“I’ve observed that at the beginning of this phase of AI, even consumers didn’t want to leverage ChatGPT or Gemini-type things because they perceived their personal privacy might be leaked. But after a while, they realize the value is much higher than expected,” said Bin.
Enterprises should make sure your infrastructure protects the valuable part of your business. It could be your data, or it could be your connection to partners. Have a clear boundary; a hybrid model is probably my recommendation. Let the bigger companies take over the model quality and the evolution of the model, while in the meantime, you know how to measure the quality of the results and you know how to organize the information.”
Brian from McKinsey added: “I think companies now are much more aware of the value of their data, and vendors are also aware of the value of their data. In fact, they think it’s probably one of the most sustainable moats they have.”
Telemetry data will drive AI forward
“Data is still the king to make AI work. Every enterprise should rethink how to capture more data for the purpose of AI,” stated Bin. “For example, this is the first time in history that a computer can learn from our behaviors. So that means in your enterprise, the tasks, how people solve their problems, how they measure the impact of a problem, should be captured somehow. You probably are missing the info. Create a data flywheel so that all the tasks you do can be captured somehow.”
Christine added, “There’s an interesting trend within the software space, where for decades, engineering teams have had streams of fairly well-formed telemetry flowing to all sorts of vendors, open source as well as SaaS. And in the last five or 10 years, we’ve had our own open standard coalition formed. It’s called OpenTelemetry, where engineering teams are able to move on to their open standard, and free themselves of vendor lock-in. I think what is happening is that those engineering teams now see the telemetry being produced by their software as theirs. They can take it to whichever vendor they choose. They can point it at whatever solutions that they want.
“How can we help those human users use that data better, perhaps with the application of AI?” Christine asked. “I love this idea of exposing the latent patterns, because humans bring context and judgment and expertise that even the best trained AIs are never going to be able to offer out of the box.”
Engineering teams are at the leading edge of the AI revolution
Christine also spoke about the impact AI has had on engineering teams everywhere:
“The thing that we’re all seeing right now with the adoption of AI tools is how rapidly the landscape is changing. Whatever pace we were operating at before, we have to become able to iterate much more rapidly and respond much more quickly. That’s placing a lot of pressure, mostly in good ways, on how engineering teams do their work.”
“You have to shift to the mindset of your employees to be aware that while AI is working, it may not always be correct. And there’s a lot of chaos that will come from deploying AI into your system. So you need to ask employees to judge whether it works, and be a new teacher to the AI system,” added Bin.
