Episode Summary – Watchman Privacy, Ep. 173: "NanoGPT: Overlord Intermediary"

Date: April 13, 2025
Host: Gabriel Custodiet
Guest: Milan Derrid, Co-founder of NanoGPT

Overview

This episode dives into NanoGPT, an innovative intermediary AI service designed for privacy-conscious users who want access to a wide array of AI models—including closed source models like ChatGPT, Claude, Gemini—without tying their identity to the big tech companies running them. Host Gabriel Custodiet and guest Milan Derrid discuss the privacy landscape of AI, compare open-source and closed-source access, practicalities of pay-per-prompt usage, censorship in models, and how NanoGPT is shaping private experimentation with generative AI. The episode is especially valuable for those seeking to explore and compare AI models without revealing personal information and while using private payment options such as Monero or Nano.

Key Discussion Points & Insights

1. What Is NanoGPT? (00:00–04:42)

• NanoGPT acts as an intermediary for users to access a variety of leading AI models—including both closed and open source—without individual accounts on each platform.
• No account required for basic use; guest access is available, with balances tied to the browser session or to an anonymous account.
• Accepts privacy-focused payment methods like Monero.
• Main privacy advantage: Your prompts are passed to AI vendors (e.g., OpenAI, Google) but without your identity or payment details.
• Quote:

  "If I was trying to design a user friendly, non coercive website, I would do it in this way ... allows you to use the service as a guest. You don't have to have an account. You can just ... use it and then exit if you would like." (A, 02:14)

Background: Milan started NanoGPT to democratize access after seeing friends blocked from ChatGPT due to payment or regional constraints.

2. Closed vs. Open Source Models; The Venice AI Comparison (04:42–07:44)

• Venice AI runs only open-source models under their control, giving extra privacy but limiting access to many cutting-edge models.
• NanoGPT enables access to both open and closed source models—via official APIs—acting as an anonymous aggregate user to the vendors.
• Limits: Prompt contents are still visible to the model providers.
• Using the API rather than web interfaces means less retention and less direct traceability.

3. Privacy, Use Cases & The Pay-Per-Prompt System (07:44–11:10)

• Use Cases: Ideal for "sampling" and experimentation, especially when you don’t want to repeatedly subscribe or create multiple accounts.
• Image models: Particularly valuable when self-hosting is not practical.
• Payment system: Pay-per-prompt, not subscription—users load a balance (via credit card or crypto) and are only charged per actual use, typically about $0.01 per query for major models, with some models costing just $0.0001/prompt and top-tier models reaching $5–10/prompt.
• Quote:

  "On average what we see is a prompt costs about $0.01." (B, 09:14)

4. Model Sophistication & Personalization (11:49–13:53)

• Quality differences: Top models excel in more complex tasks (coding, medical queries), while simple fact-finding is similar across most models.
• Different models offer unique personalities and censorship levels.
• Role playing and creative tasks: User preferences vary greatly between models like Claude, Gemini, and ChatGPT.

5. Model Access and Coverage (13:53–15:05)

• NanoGPT currently offers ~150 text models and a suite of image/video models, covering almost all mainstream and many obscure options.
• GROK (from X/Twitter) not yet available due to API restrictions.

6. Data Ownership, Privacy, and Terms (15:19–19:37)

• Generally, API use grants more generous ownership/retention terms than using the web-based subscriptions.
• NanoGPT itself stores minimal info: no IPs, cookies, conversations, or prompts (except required by Stripe if paying with credit card).
• Payments in Monero or Zcash are nearly untraceable; Nano stores sending crypto address (unless using privacy coins).
• Ownership of generated content usually belongs to the user, with less data stored by conglomerate AI companies.
• Quote:

  "So what can we see? We can see a user id ... which model ... and at what time. But we can never see the actual message ... nor do we know the IP..." (B, 16:54)

7. Accounts vs. Guest Use (19:37–21:30)

• Without an account: balances and prompt history are browser-local; not transferable between devices.
• With an account: ability to share balance across devices, using any (even temporary) email—no email verification or blocking of VPN/Tor.
• Quote:

  "If you want to be able to share your balance between say your laptop and your phone, just get an anonymous email ... use VPN on both..." (B, 19:37)

8. Cost Comparison: NanoGPT vs. Subscriptions (e.g., MidJourney) (21:30–24:55)

• NanoGPT is cost-effective for infrequent or moderate users; subscriptions only make sense with heavy or exclusive use of a single service and model.
• For most people, NanoGPT will be cheaper, more flexible for experimentation, and lower commitment.
• NanoGPT recently added MidJourney access but currently lacks two-step image prompts and extensive customization.

9. AI Privacy Abuses and Censorship (24:55–37:48)

a. Data Gathering Concerns (24:55–26:55)

• Major AI providers accumulate massive datasets combining prompts, user identity, and payment data.
• While no major leaks or abuses yet, the risk of government or commercial exploitation is substantial.

b. Prompt Isolation and Memory (27:17–28:48)

• Models have no persistent memory: history is session-bound and not stored beyond the conversation unless the browser cache is cleared.

c. AI Model Censorship & De-Censoring Techniques (31:45–37:48)

• Chinese models are strict on China-related topics but less on other areas (e.g., 3D printing firearms).
• Western models (OpenAI, Anthropic) tend to censor based on Western liberal values.
• Some censoring is coded into models during training (RLHF); open-source models can sometimes be "de-censored" by removing these layers ("obliterated" versions).
• Quote:

  "Our special trick is to just give you the models as they are ... many others add censorship layers before the model." (B, 34:00)

10. Model Recommendations and Observations (Text & Image) (37:48–41:06)

• Preferred models for Milan shift as new releases appear:
  • Text: Gemini 2.5 Pro (coding and cost-effectiveness), O1 Pro (top performance, very expensive), Claude (for human-like answers), Deepseek Reasoner.
  • Images: Reeve Half Moon (great new model; cheap, beautiful), MidJourney (high-quality, simple prompts).
• NanoGPT hosts both original and de-censored ("obliterated") versions when available.
• Some public providers (e.g., Deepseek via NanoGPT) can be used in a more privacy-preserving way by using open-source endpoints instead of Chinese servers.

11. Model API Access and Abuse Mitigation (41:06–42:43)

• NanoGPT has been mostly unaffected by abuse, as AI vendors filter most malicious content on their side.
• Occasional DDoS-like activity is handled by their hosting provider (Vercel); manual blocking has not been necessary.

12. Issues with Video Generation Models (42:43–43:50)

• Google’s VO2 video model is best for quality but extremely sensitive (“censorious”).
• If the generated video is flagged (even for benign prompts), users might be double-charged.
• NanoGPT will refund affected users who reach out, and plans to add more warnings in the UI.

13. Encrypted Storage and Syncing Plans (44:55–48:38)

• NanoGPT works on adding optional encrypted cloud-storage for conversations—default will remain local storage unless users opt in.
• Considering power-user features like custom URLs or private key logins for stateless, email-free account association (Silent Link model).
• Quote:

  "If we ever do it, it's definitely going to be like off by default ... many people really like the local storage..." (B, 44:55)

14. Nano Cryptocurrency Usage (48:38–50:57)

• Nano (currency): Used for instant, zero-fee payments; not created by NanoGPT, just adopted for operational utility.
• Nano is pseudo-anonymous (not as private as Monero); Monero is also fully supported and is the 2nd most used on the platform.

15. Web Search Integration (51:05–52:45)

• "Enable web" prepends web search results (via LinkUp API) to the model’s context, allowing it to answer current or news-related questions.
• Particularly useful for non-static or up-to-date events, as most models are trained on data that's months or years old.

16. Best Practices and Final Takeaways (52:46–54:08)

• Ultimate privacy: Run AI models locally or use open-source platforms (Venice AI, self-hosted models).
• NanoGPT's niche: Accessible, private experimenter’s tool for sampling wide AI landscape; limitations for ultra-sensitive tasks.
• Top privacy: Use Tor/Monero, do not make an account, do not input personal data.

Notable Quotes & Memorable Moments

• "I wanted to start something that could share that same magic... but make it in a way that's more accessible... So you don't need subscriptions, you don't need credit cards."
  — Milan (B), 02:49

• "All of the requests that we do to OpenAI come from just one location, one IP, one user... They don't see your name, they don't see your payment details... it adds a sort of layer of privacy at least."
  — Milan (B), 05:35

• "What can we see? We can see a user id... but we can never see the actual message that you're sending, nor do we know the IP..."
  — Milan (B), 16:54

• "Running a model locally is the best way for your privacy... But we hope that if you want to use the closed source models... we can sort of give you the best possible privacy at least."
  — Milan (B), 52:46

• "If you want to be able to share your balance between say your laptop and your phone, just get an anonymous email... use VPN on both devices. And then... you're pretty much as private as you would be..."
  — Milan (B), 19:37

• "With Deepseek, for example, we host an obliterated version... so it will gladly tell you what happened on Tiananmen Square..."
  — Milan (B), 36:38

Timestamps for Important Segments

• Intro to NanoGPT, privacy-first design: 00:00–04:42
• API-based access vs. open source/closed source: 04:42–07:44
• NanoGPT privacy: what gets shared, what gets retained: 05:35, 16:54
• Pay-per-prompt explained: 09:14–11:10
• Difference in AI model sophistication: 11:49–13:53
• Platform coverage, model availability: 13:53–15:05
• Account vs. guest workflow: 19:37–21:30
• When subscriptions still make sense: 21:30–24:55
• AI privacy abuses and model censorship: 24:55–37:48
• Favorite models and recommendations: 37:48–41:06
• API abuse prevention: 41:06–42:43
• Web search augmentation: 51:05–52:45

Tone & Style

Casual, practical, sometimes philosophical about the privacy and censorship landscape in AI. Milan is forthright about the platform’s capabilities and limitations; Gabriel interrogates the privacy nuances and opportunities for self-hosting or maximizing anonymity.

Summary Takeaway

NanoGPT is positioned as a uniquely privacy-conscious, minimalist intermediary for AI experimentation—perfect for those unwilling to sacrifice privacy to Big Tech but who still want to explore the full range of generative models. While ultimate privacy will always require self-hosting, NanoGPT offers the friendliest on-ramp for everyday explorers, learners, and researchers, with flexible payments, minimal data retention, and broad model access. Knowledgeable users can push their privacy even further by combining guest mode, Monero payments, VPN/Tor, and careful prompt management.