From DeepSeek to local installations, exploring the democratization of AI and Europe's opportunity

In this episode of Follow the Rabbit, Igor and Johannes are joined by Axel Quack, who has led the development of AI-driven products and services for large-scale organizations. Drawing from his extensive experience in financial services innovation and technology transformation, Axel brings unique insights into how European businesses, particularly SMEs, can leverage open source AI while maintaining privacy and control. His hands-on experience with local LLM installations and commitment to democratizing AI technology provides a practical foundation for exploring these emerging possibilities.

The conversation examines how recent developments like DeepSeek are challenging the narrative that AI innovation requires massive computing resources and corporate infrastructure. Through Axel's perspective as both a practitioner and strategist, the discussion reveals how smaller organizations can meaningfully engage with AI technology while maintaining their independence and data sovereignty.

The discussion weaves together several key themes:

How open source AI models are becoming increasingly accessible to individuals and smaller organizations
The tension between convenience and control in AI deployment
Europe's unique opportunity to differentiate itself through decentralized, privacy-focused approaches
The importance of hands-on experimentation in developing AI literacy
How product development might evolve as foundation models become commoditized

Through practical examples and strategic analysis, the episode reveals how the democratization of AI technology could reshape not just technical capabilities, but organizational structures and regulatory approaches. The conversation challenges both techno-optimistic and techno-skeptical positions, suggesting a more nuanced path forward that emphasizes experimentation, understanding, and thoughtful implementation.

Chapters:

00:00 - Introduction & Context

02:19 - Understanding Open Source LLMs

07:37 - Local Installation & Implementation

19:42 - European Perspectives & Opportunities

34:15 - Product Development & Future Directions

48:33 - Learning & Experimentation

54:22 - Closing Thoughts

Links & Resources:

LM Studio (local LLM installation tool): https://lmstudio.ai/
OpenWeb UI: https://docs.openwebui.com
Ollama: https://ollama.com
Ollama (supported models): https://ollama.com/library (e.g. Mistral, LlaMa, Phi, Gemma, DeepSeek, Qwen etc.)
Exo (software to run your own AI cluster on a Mac Mini): https://github.com/exo-explore/exo
Stable Diffusion: Automatic1111 (UI) - https://github.com/AUTOMATIC1111/stable-diffusion-webui
Smart2Brain: AI plugin for Obsidian based on Ollama - https://github.com/your-papa/obsidian-Smart2Brain
Zima Cube: https://zimacube.com/

This episode provides both practical insights for those interested in experimenting with local AI installations and broader strategic perspectives on how open source AI might reshape technological development. It suggests that while we should maintain healthy skepticism about AI hype, the democratization of these tools creates new opportunities for innovation and agency.

Running AI on Your Terms: Open Source LLMs & European Innovation