AI Product Managers
AI Product Managers
AI and Product Management, 10x engineers, Greece’s AI landscape, state of Ethereum infrastructure, jobs, events, and more
Friends, welcome back to Startup Pirate, a newsletter about what matters in tech and startups with a Greek twist. Come aboard and join 4,908 others. Still not a subscriber? Here’s what you recently missed:
AI Product Managers
What are the main challenges teams face when building AI products? What’s the right team structure? How to build an AI MVP? What changes in how teams measure product success? How can Product Managers lead through technical influence?
This week, we have Marily Nika, a computer scientist and AI Product Leader who has worked for Google & Meta’s Reality Labs for over ten years, to discuss AI and product management. Previously, Marily completed a PhD in Machine Learning at Imperial College London. She is also an Executive Fellow at Harvard Business School and actively teaches and writes about how Product Managers can thrive in AI.
Let’s jump in 👇
Marily, thanks for being here. I’ve followed your work online, and something that caught my attention was your thesis about how AI will be baked into every software product and how Product Managers will be AI Product Managers.
MN: Thank you for having me, Alex. Indeed, I believe all Product Managers will be AI Product Managers because, in the future, every solution will have smart features powered by Machine Learning. Hence, PMs need to have at least a basic understanding of AI and the AI lifecycle, all the way from scoping to collecting data to modelling, testing, and what it takes to productionise such technology. Moreover, certain soft skills are key, such as how to work with scientists and engineers to build successful products.
What are the main challenges for teams that start building AI products and pose a significant departure from their previous modus operandi?
MN: When traditional organisations, used to a standard setup of software engineers, a product manager, and a designer, venture into AI product development, they encounter a whole new world of challenges. What got us here won’t get us there — which means that what worked in the past doesn’t necessarily pave the way forward in the AI world. The shift from a culture of inspiration to one of experimentation is monumental.
In my experience, I have seen teams work for months on an AI solution, only to find it unusable due to poor quality in the experience they power. This can be a morale crusher, and it falls on the PM to keep the team motivated and focused on the broader vision.
The transition from a deterministic approach to an experimental one is crucial. Quick pivots should become the norm. Decision-making, especially for a PM, gets trickier. With AI, there’s rarely a polished solution on day one. Critical questions arise that PMs need to have an opinion on/answer: What kind of data does our model need? Do we have enough? If not, can we gather or synthesise more? These decisions aren’t just technical; they are central to the product’s direction.
Many teams get caught up in the “shiny object trap,” pursuing AI for its “prestige” rather than utility. But sometimes, simple hardcoded rules can be quicker and more cost-effective than the complex AI training process, especially for validating a product idea. This approach can often yield a “good enough” result without the overhead of AI development. Think of it as an AI MVP.
Your strategy should drive your approach. If you’re a startup looking to validate an idea quickly, put together different pieces and perhaps leverage AI services through APIs. If you’re a larger tech company aiming to build a bespoke solution, clearly define the Minimum Viable Quality (MVQ). This is the threshold at which a product’s experience is sufficient for a public launch. Can you achieve this with no-code tools, or do you need to build from the ground up?
Remember, AI isn’t a product in itself; it’s a tool to solve user problems. The choice of how to use this tool should align with your overarching business strategy and user needs.
Are the metrics AI Product Managers use to measure product success different?
MN: When it comes to AI product management, the metrics we use to measure success can be quite different from traditional methods. Success is a blend of metrics, as I like to call it. Product Health, AI proxy and System Health.
Product health: engagement, retention, monetisation, etc. These are the typical metrics most Product Managers use and relate to the company’s strategic goals.
AI metrics: several metrics can be used to detect how accurate an AI model is, for instance, the fraction of predictions our model got right, the percentage of false positives and false negatives, etc. There are specific terms like accuracy, precision, recall, etc. This is where your AI strategy comes in, where you make trade-offs between being the first to market and perfecting how your AI system works. A good example is Apple. Apple always comes in with the best quality, and its whole premise is to have pristine design and polished experiences. But often, they are not the first to market a new product category.
System performance metrics: Traditionally, these are not related to product management, but in AI, the PM needs to ensure that as user numbers grow, the AI model continues to perform well without compromising the user experience. As you get more users, the model can still perform well, and the experience does not break. Examples include system response time, server uptime, and the efficiency of data processing.
What skills do you think AI Product Managers should spend time developing?
MN: I would say there are two main areas. First, technical influence:
While PMs may not be directly building AI models, having a solid grasp of the technical aspects is vital. This knowledge enables them to make informed strategic trade-offs, like balancing accuracy with delivery speed or personalisation with privacy concerns.
Understanding the language of AI and its workings allows PMs to collaborate with AI scientists and engineers effectively. It’s this technical influence that empowers PMs to inspire their teams and lead effectively, embodying what we call “influence without authority.”
Second, strategic leadership:
Beyond the hustle of implementation and team management, being a PM in the AI space demands strategic foresight and leadership. It’s about being a visionary leader who can chart a course for the product and instil trust in the team.
This strategic aspect involves looking beyond day-to-day operations and focusing on long-term goals and visions. It’s about steering the product in a direction that aligns with broader market trends and user needs.
Before we wrap up, shifting gears for a moment. You briefly worked on the Codec Avatars project at Meta, the company’s new photorealistic virtual reality avatars. How do you see this technology evolving over the next few years?
MN: I’m fascinated by the future of virtual reality and being able to connect with friends, family, and colleagues when you’re not in the same place. Meta is a pioneer in this space, with many leading scientists working on hardware and software that enables such authentic experiences. What you saw a few months ago in this great interview of Mark Zuckerberg and Lex Fridman was the first time people experienced an interview in the metaverse.
The progress we are experiencing in this field nowadays is due to a combination of hardware, software, and, of course, a desire to build breakthrough technologies that change the course of human history. As the next-gen of mixed reality headsets like the new Meta Quest Pro comes to market, as we acquire more and more data from human faces to train the models, and as AI models improve over time, we will soon be able to experience a whole new level of immersive social experiences.
Thank you so much for taking the time, Marily!
MN: Appreciate it, Alex.
Jobs
Check out job openings here from startups hiring in Greece. The list now has 537 jobs from 94 companies.
News
iSIZE Technologies, a company specialising in deep learning for video delivery, was acquired by Sony Interactive Entertainment. (link)
Digital payments OKTO acquired a US fintech startup to accelerate market expansion into the North American gaming markets. (link)
Multiwave Technologies secured CHF 3.8m for its light portable MRI device. (link)
Zephr raised $3.5m Seed to launch a networked GPS solution that improves location accuracy by using nearby mobile phones as base stations that provide error correction. (link)
Drone company ALTUS LSA (link) and tech & digital agency SLEED (link) raised funding.
Envolve accepting applications for its startup acceleration program until 10 January 2024. (link)
New Products
Free Tools for Managers by Workable, Sentify AI (monitoring and analytics for LLM products), Global HR (HR for crypto organisations)
Resources
Building a 3D printing business in the 1990s, investing in hardware, the impact of AI on productivity, and more with Marina Hatsopoulos, entrepreneur & investor. (link)
George Hadjigeorgiou, co-founder & President of ZOE, on building the science before seeking funding and treating culture as a product. (link)
Mikri Kouventa episode with Paris Kasidiaris and Antonis Kalipetis discussing 10x engineers, engineering productivity, and building a great team. (link)
A deep dive into Greece’s AI landscape from The Greek Analyst. (link)
John Gianakopoulos, Principal at Scale Venture Partners, on finding early success with vertical SaaS LLMs. (link)
Engineering best practices for data science by Matina Thomaidou, VP Data Science at Dataseat. (link)
Elias Simos, founder & CEO of Rated, on the state of Ethereum infrastructure. (link)
Joseph Alvertis, VP Product at TileDB, on being pragmatic with product management. (link)
The truth about your employability as a manager by Zaharenia Atzitzikaki, Design leader & Executive coach. (link)
Events
“12th Open Coffee Mytilene” by Open Coffee Mytilene on Nov 20
“From EX to CX: The Ripple Effect of Employee Engagement” by CX Greece on Nov 29
“Attribution Modelling: From Interaction to Action” by Business Intelligence and Analytics Athens on Nov 30
“Building High-Performance Microservices with Go” by Goland Athens on Nov 30
“Greeking Out in New York” by Endeavor Greece on Dec 4
“AthensDAOx 2023” on Dec 9
If you liked this piece, I’d be grateful if you’d consider tapping the heart ❤️ below. It helps me understand which pieces you like best, and what I should do more.
Find me on LinkedIn or Twitter.
Until next time,
Alex
 | A guest post by
|