Podcast Summary: CoRecursive – Tech Talk: GraphQL with Oleg Ilyenko

Host: Adam Gordon Bell
Guest: Oleg Ilyenko, creator of Sangria (GraphQL for Scala)
Date: April 18, 2018

Episode Overview

This episode explores the world of GraphQL through an in-depth conversation with Oleg Ilyenko, the creator of Sangria, a GraphQL implementation for Scala. Host Adam Gordon Bell discusses common API challenges, how GraphQL addresses them, the strengths and weaknesses of both REST and GraphQL, and gets into the technical nitty-gritty of building APIs with Sangria.

Key Topics & Insights

1. The Problem with Traditional APIs

• Fast-Evolving Data Models: Adam shares his struggles with traditional REST endpoints as client data needs evolve (00:32-02:21).
• Challenges in Versioning: Adding new fields or supporting different clients often leads to versioning headaches and complex endpoint management.

  “You just want to change one field or want to remove one field. So you need to be very cautious about introducing new changes...” – Oleg (08:23)

2. What Makes GraphQL Different?

• Query Flexibility: Clients specify exactly what data they require, avoiding over-fetching and under-fetching (03:46-04:09).

• No More Global Versioning: Instead of versioning the whole API, GraphQL focuses on the shape of the response clients need (04:09-05:33).

  “With GraphQL, you no longer think in terms of the version, you're thinking in terms of data requirements.” – Oleg (04:09)

• Field Deprecation & Usage Tracking:

  • Fields can be marked as deprecated, and usage is tracked so server maintainers know when it’s safe to remove them (05:49-08:07).
  • Real production insight: Oleg describes using InfluxDB and Grafana to see which clients are still using deprecated fields.

3. Overfetching & Custom Field Selection

• REST Solutions: Sometimes, REST APIs add mechanisms for clients to select fields. But standards vary and nesting becomes awkward (10:39-13:35).
• GraphQL’s Approach:
  • Uniform, introspectable specification lets any client specify fields and traverse relationships, with language support for nested queries.
  • All GraphQL APIs work the same way, allowing tooling and discoverability.

4. The Origins and Strength of GraphQL’s Data Model

• GraphQL Origin: Initially designed for Facebook’s mobile API needs in 2012 (13:57-15:20).
• Graph vs. Resource Modeling:
  • REST tends toward isolated resources, while GraphQL models connected data as a graph (16:13-17:23).
  • Oleg: “The conceptual model is a graph... types and relationships instead of tables and foreign keys, or resources and links.” (15:20)

5. What Is GraphQL? (Specification vs. Implementation)

• GraphQL = Specification:
  • Not just a library—GraphQL is a spec, allowing many languages to implement it uniformly (18:34-20:07).
  • “GraphQL is a specification, which you can think of it as just a PDF document describes as a syntax semantic.” – Oleg (18:34)
• Not a Database Frontend:
  • GraphQL is not like SQL; it's up to server developers to write the evaluators, which fetch and assemble data from any backend (20:07-22:16).
  • Example: Multi-backend fetching (MongoDB, Elasticsearch, internal APIs, etc.).

6. Introducing Sangria (Scala GraphQL Implementation)

• Why Sangria:
  • Oleg created Sangria to bring GraphQL to Scala after being frustrated by REST APIs (24:26-26:38).
• Leveraging Scala’s Type System:
  • Fits naturally with Scala’s type class modeling; GraphQL exposes a type system, though not as expressive as Scala’s (26:45-28:30).

7. Strong Typing & Introspection

• Introspective APIs:
  • Users can query a GraphQL API for its own schema and documentation (29:07-31:03).
  • Mutations & input types enable full CRUD, not just read access.

8. Implementation Details with Sangria

• Easy Schema Building:

  • Using explicit object types or macros for automatic schema generation (36:22-38:42).

• Custom Fields & Batch Loading:

  • Custom resolvers allow computed fields or remote data fetching (41:27-42:10).
  • Deferred/batch loading solves the "N+1 query problem" by collecting keys and fetching in bulk as needed (43:59-46:53).
  • Oleg: “At the end of the day... you will end up with at most three SQL queries or maybe HTTP requests to internal microservices.” (46:53)

• Separation of Data Model and API Model:

  • You can keep internal and external (API) models separate, avoiding overexposure (40:40-41:00).

9. Scale & Adoption

• Real-World Use at Scale:
  • Twitter, New York Times, and Coursera use Sangria, with Twitter reportedly doing 2 billion requests per day via Sangria (35:07).

    “Last time as I tweeted about it... it was about 2 billion requests per day at the moment.” – Oleg (35:07)

• Feedback Loops:
  • Open-source benefit: feedback and contributions from major companies improve the library (33:37-34:56).

10. When NOT to Use GraphQL

• REST is Still Relevant:
  • For highly cacheable, stable APIs (e.g., Wikipedia), REST may be better due to tight HTTP coupling and mature caching infrastructure (49:28-51:08).

    “In this case I think REST API might fit much better because it's so tightly coupled with the HTT and the way HTTP works.” – Oleg (49:28)

11. Common Pitfalls and Learning Curve

• Different Mental Model:
  • Developers new to GraphQL may cling to resource-centric thinking and not leverage relationships in the data model (52:00-52:54).
• Built-in Discoverability:
  • Self-describing APIs enable better tooling and schema diffing for change management (53:20-54:45).

    “If you can have a GraphQL endpoint, just a URL, you know precisely how you will figure out like what types it provides, what things you can do with it.” – Oleg (53:41)

Notable Quotes

• On Field-Level Deprecation & Safety:

  “Given this, and given that server always knows what clients uses, we can track the usage of particular fields... We can define thresholds, we can define the time frame, for how long do we wait until we drop this field. But when we drop this field, we have a lot of confidence that this field is not used by any client.” – Oleg (07:10)

• On Discoverability and Tooling:

  “All GraphQL APIs work in exactly the same way... you can build very powerful tools, especially considering that GraphQL API also has a full introspection exposed.” – Oleg (13:12)

• On Batch/Bulk Loading:

  “Instead of loading an organization, you can... return a deferred value. What execution engine does, it collects all of those deferred values and... can very, very efficiently load all this data at once.” – Oleg (43:59)

• On Open Source Responsibility:

  “I was very excited and maybe a little bit scared because, you know, you never know, like maybe something is wrong and something will cause disaster.” – Oleg on Twitter’s Sangria adoption (33:37)

Timestamps for Key Segments

| Timestamp | Segment Description | |-----------|--------------------| | 00:32–02:21 | Adam describes API versioning problems | | 03:46–05:33 | Oleg explains GraphQL's approach to field selection and avoiding breaking changes | | 05:49–08:07 | Field deprecation and usage monitoring in GraphQL | | 13:57–16:13 | History of GraphQL at Facebook & graph data modeling | | 18:34–20:07 | GraphQL as a specification, implementations across languages | | 24:38–26:38 | Motivation and process behind creating Sangria | | 29:07–31:03 | Exploring the introspection API and complex mutations | | 43:57–46:53 | Batch data loading and avoiding "N+1" problem | | 49:28–51:08 | Situations where REST is a better choice than GraphQL | | 53:20–54:45 | Schema diffing and tooling enabled by introspection |

Memorable Moments

• Oleg’s Reaction to Twitter using Sangria:

  “I was very excited and maybe a little bit scared because, you know, you never know... But it was a while now and… they actually contributed some of the improvements to the library.” (33:37)

• Production Scale:

  “It was about 2 billion requests per day at the moment.” (35:07)

• Macro-based Schema Generation:

  “You can derive the structure of a case class, but you can still provide a description for fields, or you can exclude or include particular fields. You can even replace field or add new fields…” (38:59)

Conclusion

This conversation gives a comprehensive, nuanced perspective on GraphQL’s strengths, trade-offs, and real-world use, both at small and massive scale. Oleg’s insights into how GraphQL (and Sangria) empower type-safe, client-driven APIs, while acknowledging REST’s continued relevance, provide valuable guidance for anyone designing modern APIs.

For further details, live demos, and production stories, tune into the full episode!