APIs & Underlying Protocols Lessons · Semantics · Trends @andreisavu #APICraftSF
Me Working on Cloudera Director Previously founder of Axemblr.com and Java User Group organizer More on LinkedIn & @andreisavu
Goal Share the idea: There is no silver bullet and any implementation must ack network laws From my experience with: ● using multiple cloud provider APIs ● using distributed systems (Zookeeper) ● writing clients & APIs (jclouds, Director)
A bit of context On the role of APIs & Web Evolution
http://techblog.netflix.com/2015/02/a-microscope-on-microservices.html
How we build modern applications? As hybrids Start: as monolithic systems that use external APIs (identity, analytics, payments, email etc.) Evolve: collections of services with cohesive APIs (map to organizational boundaries)
What is the role of an API? Message passing protocol for State Machines Often seen as replacing libraries in monolithic systems. Dangerous: doesn’t account for network semantics (reason why RPC failed)
What is the role of an API? Message passing protocol for State Machines Better: Writing and reading messages on a synchronous queue (producer / consumer). Matches transport layer semantics.
Fashion-Driven-Development Are you done rewriting your app in / on X!?
Reinventing the wheel? Often too young to know better? (simple vs. simplistic approaches)
Doing CORBA again?! (1991)
Good API / Bad API acknowledge the distributed nature of the app actual choice of technology is orthogonal
Network is not reliable Partial failures, Timeouts, Partitions, At-least once message delivery
Latency is not zero Or predictable. Jitter. Stranglers.
Bandwidth is limited Fanout. Object size. Compression. Capacity
The network is not secure Identity, Authorization, Encryption, Throttling
Topology does change Endpoint discovery. Versioning, DNS
Transport cost is not zero CPU, Contention, Syscalls, GC
There are multiple operators A system controlled by multiple entities
Network is heterogeneous No one size fits all
Your API is as good as your client library Users writing their own doesn’t really work (especially for static languages)
Not all protocols are equal Wide adoption has important network effects Gravity. Fashion vs. Technical Merits
How did REST-like succeed? Mix: HTTP, Thin Spec, Partial Adherence Piggyback on: CDNs, Proxies, Tooling etc.
HTTP/1 → HTTP/2
HTTP/2: new framing layer maintains HTTP paradigm same URL structure (including TLS) designed for 1:1 conversion at proxy allows for server push http://daniel.haxx.se/http2/
APIs better suited for: ● low cost batch operations ● lower latency ● compatibility with next gen CDNs ● downside: more complex clients … but the devil is in the details. What we can hope w/ HTTP/2?
Take aways There are no silver bullets There has to be substance behind fashion Rewriting everything is the norm Acknowledge transport layer restrictions Technology as a mean-to-an-end
Thanks! Questions? asavu@apache.org @andreisavu

APIs & Underlying Protocols #APICraftSF

  • 1.
    APIs & UnderlyingProtocols Lessons · Semantics · Trends @andreisavu #APICraftSF
  • 3.
    Me Working on ClouderaDirector Previously founder of Axemblr.com and Java User Group organizer More on LinkedIn & @andreisavu
  • 4.
    Goal Share the idea:There is no silver bullet and any implementation must ack network laws From my experience with: ● using multiple cloud provider APIs ● using distributed systems (Zookeeper) ● writing clients & APIs (jclouds, Director)
  • 5.
    A bit ofcontext On the role of APIs & Web Evolution
  • 9.
  • 10.
    How we buildmodern applications? As hybrids Start: as monolithic systems that use external APIs (identity, analytics, payments, email etc.) Evolve: collections of services with cohesive APIs (map to organizational boundaries)
  • 11.
    What is therole of an API? Message passing protocol for State Machines Often seen as replacing libraries in monolithic systems. Dangerous: doesn’t account for network semantics (reason why RPC failed)
  • 12.
    What is therole of an API? Message passing protocol for State Machines Better: Writing and reading messages on a synchronous queue (producer / consumer). Matches transport layer semantics.
  • 13.
    Fashion-Driven-Development Are you donerewriting your app in / on X!?
  • 14.
    Reinventing the wheel? Oftentoo young to know better? (simple vs. simplistic approaches)
  • 15.
  • 16.
    Good API /Bad API acknowledge the distributed nature of the app actual choice of technology is orthogonal
  • 17.
    Network is notreliable Partial failures, Timeouts, Partitions, At-least once message delivery
  • 18.
    Latency is notzero Or predictable. Jitter. Stranglers.
  • 19.
    Bandwidth is limited Fanout.Object size. Compression. Capacity
  • 20.
    The network isnot secure Identity, Authorization, Encryption, Throttling
  • 21.
    Topology does change Endpointdiscovery. Versioning, DNS
  • 22.
    Transport cost isnot zero CPU, Contention, Syscalls, GC
  • 23.
    There are multipleoperators A system controlled by multiple entities
  • 24.
    Network is heterogeneous Noone size fits all
  • 25.
    Your API isas good as your client library Users writing their own doesn’t really work (especially for static languages)
  • 26.
    Not all protocolsare equal Wide adoption has important network effects Gravity. Fashion vs. Technical Merits
  • 27.
    How did REST-likesucceed? Mix: HTTP, Thin Spec, Partial Adherence Piggyback on: CDNs, Proxies, Tooling etc.
  • 28.
  • 29.
    HTTP/2: new framinglayer maintains HTTP paradigm same URL structure (including TLS) designed for 1:1 conversion at proxy allows for server push http://daniel.haxx.se/http2/
  • 30.
    APIs better suitedfor: ● low cost batch operations ● lower latency ● compatibility with next gen CDNs ● downside: more complex clients … but the devil is in the details. What we can hope w/ HTTP/2?
  • 31.
    Take aways There areno silver bullets There has to be substance behind fashion Rewriting everything is the norm Acknowledge transport layer restrictions Technology as a mean-to-an-end
  • 32.