Lessons Learned from Building a REST API on Google App Engine

Lessons Learned from Building a REST API on Google App Engine

• 1.
  Lessons Learned fromBuilding a REST API on Google App Engine Jonathan Altman Presentation to GolangDC-October 29, 2015
• 2.
  Whitenoise Market Webapp •White Noise by TMSoft (http://www.tmsoft.com/white-noise/) is the leading sleeping app for iOS,Android, Mac, and Windows • Customer wanted a way to: • Allow users to download additional content to the app • Create a vibrant community for users to interact with each other • Scale to the large demand of existing users
• 4.
  White Noise MarketApp
• 5.
  Project • Build aRESTful API to drive Whitenoise Market’s web front-end • Angular SPA front end, also built as part of the project • User authentication with Google or Facebook account—OAuth2 • Role-based authorization • Implied: customer will use the API from a native mobile client as well • Golang on Google App Engine, leverage their APIs
• 6.
  Sample Calls • GET/api/items — get all items • GET /api/item/item_id — get data about the item with id item_id
• 7.
  GAE via Golang •Project was approx. 6 person/weeks 2nd 1/2 2014, including front end • Customer specification based on their research • Inherited solid proof of concept app, but no firm API • GAE golang support was still beta, long term support indeterminate • Actual GAE API usage calls: outside the scope of this talk (but see https://cloud.google.com/appengine/docs/go/)
• 8.
  Issues • Package management •Routing • REST response formulation/error logging • OAuth2 support for providers other than Google • Authorization • Miscellaneous
• 9.
  Package Management • goappget not go get • Not building an exe locally, packages need to be in source tree uploaded to GAE - feels weird compared to golang philosophy
• 10.
  Routing — GAEhas choices • Prefix hostname with module — exposing internals • Dispatch file: dispatch.yaml — 10 routing rules max • Roll your own — just start matching URLs in the main dispatch handler in your golang code • or… • and remember: Google Cloud Endpoints were not yet a thing. Probably the way to go today
• 11.
  RollYour Own Router
• 12.
  3rd Party Router:Gorilla mux! • http://www.gorillatoolkit.org/pkg/mux • Gorilla web toolkit has a bunch of other nice parts • Other 3rd party router libraries probably work fine • Parameterization, method control • GAE takes care of a lot of other things Gorilla toolkit provides r.HandleFunc("/api/comments/{sid}",  handleGetComments).Methods("GET")
 r.HandleFunc(“/api/comments/{sid}",  aihttphelper.AuthenticatedEndpoint(HandleAddComment)).Methods("PUT")
• 13.
  REST Status/Response Logging •Standard REST success and error responses • gorca — https://github.com/icub3d/gorca • gorca.LogAndMessage: Logs console message and returns short message plus status code • gorca.WriteJSON: succesful responses gorca.LogAndMessage(c,  w,  r,  err,  "error",  "not_authenticated",  http.StatusUnauthorized)   gorca.LogAndMessage(c,  w,  r,  err,  "error",  err.Error(),  http.StatusBadRequest)   gorca.WriteJSON(c,  w,  r,  map[string]interface{}{“status”:  "OK",  "tagAdded":  tagValue})  
• 14.
  OAuth2 Support -gomniauth • GAE does OAuth2 authentication…only for Google • gomniauth does OAuth2 authentication for multiple providers, including google (https://github.com/stretchr/gomniauth) • jwt for HTTP Bearer Token — (https://github.com/dgrijalva/jwt-go) • Accepted pull request in gomniauth allows setting http Transport used because the GAE runtime replaces net/http’s DefaultTransport with a context-based one https://github.com/stretchr/gomniauth/pull/23)
• 15.
  gomniauth Patch • Youhave to fetch a Transport with the current requests’ GAE context, and pass that to gomniauth before doing authentication • See https://github.com/jonathana/gomniauth/commit/ 3e2e23995b035e26bbd58a0f56cb2b2d61dbe993 for details/usage
• 16.
  Authorization • Separate fromauthentication. What a user can do, once we know who the user is • Wrapper function shown before: • “Middleware” takes a target function with an extra argument beyond the normal HTTP request handler for the authenticated user information, and returns a normal HTTP handler function that does the authorization check and runs the target function if authorized • Factory functions encapsulated role info, but could pass in ACL data r.HandleFunc(“/api/comments/{sid}",  aihttphelper.AuthenticatedEndpoint(HandleAddComment)).Methods("PUT")
• 17.
  Authorization Middlewaretype  AiHandlerFunc  func(appengine.Context,  http.ResponseWriter,  *http.Request,  *aitypes.AIUserInfo)   func  generateAuthenticatedEndpoint(h  AiHandlerFunc,  requiredRoles  aitypes.RoleValue)  http.HandlerFunc  {
   return  func(w  http.ResponseWriter,  r  *http.Request)  {
     c  :=  appengine.NewContext(r)
   
     authUser,  err  :=  AuthenticateRequest(c,  r)
     if  (err  !=  nil)  {
       gorca.LogAndFailed(c,  w,  r,  err)
       return
     }
     //  401  User  not  authenticated     if  (authUser  ==  nil)  {
       http.Error(w,  "",  http.StatusUnauthorized)
       return
     }
     //  403  User  not  authorized  (authenticated,  but  no  permission  to  resource)
     if  (requiredRoles  >  0  &&  !(hasRole(authUser,  requiredRoles))  {
       http.Error(w,  "",  http.StatusForbidden)
       return
     }
   
     //  User  is  authenticated  and  authorized
     h(c,  w,  r,  authUser)
   }
 }   func  AuthenticatedEndpoint(h  WnHandlerFunc)  http.HandlerFunc  {
   return  generateAuthenticatedEndpoint(h,  0)
 }
• 18.
  Miscellaneous • Concurrency: ignoredas a premature optimization. Issues with urlfetch.Transport led to concern on runtime support/research time • GAE API deprecation: not golang specific, but several APIs in use were deprecated post-project and had to be replaced (blobstore) • GAE appears to be going to more of an a la carte model where existing components are replaced with general GCE equivalents • Google Cloud Endpoints were not available at the time
• 19.
  Miscellaneous, cont. • You’llbe playing with the JSON serialization properties. Javascript<- >go naming rules mismatch: nobody wants Javascript properties to begin with capital letters. Also, I tend to prefer map[string]interface{} over defined structs where I can • Using appengine.Context. You will need to, almost everywhere, whether it’s for working with datastore, making outbound http requests, or logging via its .Infof() call
• 20.
  ThankYou! email: jonathan@async.io github: jonathana twitter:@async_io