RESTful webservices with Python For lazy developers or developers with deadlines
source: http://s3-ec.buzzfed. com/static/enhanced/webdr01/2012/12/2/13/e nhanced-buzz-wide-18383-1354473319-2.jpg
Who am I? Justyna Żarna Woman in Django / Python World JavaScript freak @Ustinez http://solution4future.com
Content 1. REST software architecture. 2. Benchmarks - problem analysis. 3. Flask + SQLAlchemy. 4. Django & class-based views. } case study
API XML/RPC SOAP REST based on HTTP based on HTTP protocol protocol or other protocols many restriction on data types synchronous ??? synchronous based on XML and all xml defects
What is so cool? REpresentational State Transfer } Resource http://a.com/resources/ http://a.com/resources/item1/ http://a.com/resources/item2/ http://a.com/resources/item3/ Representation Representation Representation GET PUT POST DELETE THE VERBS
What is so cool? REpresentational State Transfer 1. Scalability 6. In HTTP context but not limited to this protocol. 2. Generalization of interface 7. Good designed interface. 3. Independence 4. Security 5. Simplicity
http://www.2010theyearinbooks.com/2012/12/december-beach-reads-for-australian.html make your API RESTful and go to rest...
Benchmark JSON response for frameworks and languages. Python frameworks: * Django-stripped 13 269 per second * Flask - 11 506 per sec * Django - 7122 per sec http://www.techempower.com/blog/2013/04/05/frameworks-round- 2/
The source code and problems https://github.com/TechEmpower/FrameworkBenchmarks/ http://www.techempower.com/blog/2013/04/05/frameworks-round-2/ In this souce code for flask and django isn't used connection pool - cache of database connection. connection connection database connection database connection connection for each request is connection are maintained for open and closed. future
Solutions? 1. Database optimalization for PostgreSQL: ● pgBouncer ● pgPool http://www.askthepony.com/blog/2011/07/django-and-postgresql-improving-the-performance-with-no-effort-and-no-code/
Solutions? code profiler can tell us more... 2. JSON standard serialization in Python STD library is slow, so we can improve performance by using module ujson - ultra json.
Solutions? 3. Rendering template can be faster with template engine jinja (inspired by Django templates)
Flask ● "micro" does not means micro possibilities, ● core is light, simple and extensible, ● support many extensions in each layer (for example your database layer can by relational database or non-relational data persistence).
Case study - micro framework Flask from sqlalchemy import Column, Integer, String from newsletter.database import Base class Member(Base): __tablename__ = 'newsletter_members' id = Column(Integer, primary_key=True) last_name = Column(String(50)) first_name = Column(String(120)) email = Column(String(120), unique=True) def __init__(self, last_name=None, first_name=None, email=None): self.last_name = last_name self.first_name = first_name self.email = email def __repr__(self): return '<Member %r>' % (self.last_name)
Case study - micro framework Flask class API(MethodView): def get(self, member_id): if member_id is None: return Member.query.all() else: return Member.query.filter_by(id = member_id).first() def post(self, data ): member = Member(first_name = data['first_name'], email=data['email']) db.session.add(member) db.session.commit() return 'OK' app.add_url_rule('/users/<int:user_id>', view_func=API.as_view('user_api'), methods=['GET', 'POST']) #class flask.views.MethodView and recognizing each REST methods are based on generic dispatch_request()
Case study - micro framework Flask It was simple and quick to code? improve simplicity and spent time with... Flask RESTless
Flask RESTless - full script import flask import flask.ext.sqlalchemy import flask.ext.restless app = flask.Flask(__name__) app.config['DEBUG'] = True app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db' db = flask.ext.sqlalchemy.SQLAlchemy(app) class Member(db/Model): __tablename__ = 'newsletter_members' Too simple for real project.. ? id = db.Column(Integer, primary_key=True) last_name = db.Column(String(50)) first_name = db.Column(String(120)) email = db.Column(String(120), unique=True) db.create_all() manager = flask.ext.restless.APIManager(app, flask_sqlalchemy_db=db) manager.create_api(Member, methods=['GET', 'POST']) app.run()
Flask RESTless - more sugar ● versioning: GET /api/member GET /api/member/(int: id) GET /api/member?q=<json> POST /api/member apimanager.create_api(Member, url_prefix='/api/v2') GET /api/v2/member ● validation: manager.create_api(Member, methods=['GET', 'POST'], validation_exceptions= [ValidationError]) { "validation_errors": { "email": "Email is not valid..", } }
Flask RESTless - more sugar ● specify columns (include or exclude): apimanager.create_api(Member, include_columns = ['last_name', 'email']) ● pagination: manager.create_api(Member, methods=['GET', 'POST'], results_per_page=2) { "validation_errors": { "age": "Must be an integer", } { "num_results": 6, "total_pages": 3, "page": 1, "objects": [ {"last_name": "Kovalsky", "email": "kovalsky@gmail.com", "id": 1}, {"last_name": "Novak", "email": "novak@gmail.com", "id": 2} ] }
Flask RESTless - more and more.. ● pre/post-processors: def pre_get_single(instid): # do something pass def pre_get_many(params): # do something pass # Create an API for the Member model. manager.create_api(Person, methods=['GET', 'POST'], # A list of preprocessors for each method. preprocessors={'GET_SINGLE': [pre_get_single], 'GET_MANY': [pre_get_many],}) ● Authentication: def auth_func(params): if not current_user.is_authenticated(): raise ProcessingException(message='Not authenticated!') return NO_CHANGE manager.create_api(Person, preprocessors={'GET_SINGLE': [auth_func]})
Flask RESTless - more and more.. ● filtering: import requests import json url = 'http://127.0.0.1:5000/api/member' headers = {'Content-Type': 'application/json'} filters = [dict(email='email', op='like', val='%y%')] params = dict(q=json.dumps(dict(filters=filters))) response = requests.get(url, params=params, headers=headers) GET /api/member?q={"filters":[{"name":"email", "op":"like", "val": "kovalsky"}]} OPERATORS examples: ○ ==, eq, !=, neq, ○ >, gte, <, lte FOR RELATIONS: ○ in, not_in ● column__column example: member__group ○ is_null ○ like ○ has ○ any
What about Django? ● Define your model: from django.db import models class Member(models.Model): last_name = models.CharField(max_length = 100, verbose_name = "Last name") first_name = models.CharField(max_length = 100, verbose_name = "First name") email = models.EmailField(max_length = 100, verbose_name = "Email") def __unicode__(self): return self.email class Meta: verbose_name = "Newsletter member" verbose_name_plural = "Newsletter members"
What about Django? ● Define your model serializer: class MemberSerializer(serializers.ModelSerializer): class Meta: model = Member fields = ('last_name', 'first_name', 'email') ● Working with serializers: from newsletter.models import Member from newsletter.serializers import MemberSerializer from rest_framework.renderers import JSONRenderer from rest_framework.parsers import JSONParser member = Member(last_name='Kovalsky', first_name= 'Johny', 'email' = 'kovalsky@gmail. com') member.save() serializer = MemberSerializer(member) content = JSONRenderer().render(serializer.data) # content: {"pk": 2, "last_name": "Kovalsky", "first_name="Johny", email = "kovalsky@gmail.com"}, more object in format [{'foo': 'bar'}, {'foo': 'bar'}]
What about Django? ● Define your class-based views: class MemberDetail(APIView): """ Retrieve, update or delete a member instance. """ def get_object(self, pk): try: return Member.objects.get(pk=pk) except Member.DoesNotExist: raise Http404 def get(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member) return Response(member.data)
What about Django? ● Define your class-based views: def put(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member, data=request.DATA) if serializer.is_valid(): serializer.save() return Response(serializer.data) return Response(serializer.errors, status=status. HTTP_400_BAD_REQUEST) def delete(self, request, pk, format=None): member = self.get_object(pk) member.delete() return Response(status=status.HTTP_204_NO_CONTENT)
What about Django? ● Or generic views: from newsletter.models import Member from newsletter.serializers import MemberSerializer from rest_framework import generics class MemberList(generics.ListCreateAPIView): model = Member serializer_class = MemberSerializer class MemberDetail(generics.RetrieveUpdateDestroyAPIView): model = Member serializer_class = MemberSerializer ● URLs patterns: urlpatterns = patterns('', url(r'^members/$', views.MeberList.as_view()), url(r'^members/(?P<pk>[0-9]+)/$', views.MemberDetail.as_view()), )
RESTful API - done http://sarahwmackey.files.wordpress.com/
Thank you for your attention http://solution4future.com/

Python RESTful webservices with Python: Flask and Django solutions

  • 1.
    RESTful webservices with Python For lazy developers or developers with deadlines
  • 2.
  • 3.
    Who am I? Justyna Żarna Woman in Django / Python World JavaScript freak @Ustinez http://solution4future.com
  • 4.
    Content 1. REST softwarearchitecture. 2. Benchmarks - problem analysis. 3. Flask + SQLAlchemy. 4. Django & class-based views. } case study
  • 5.
    API XML/RPC SOAP REST based on HTTP based on HTTP protocol protocol or other protocols many restriction on data types synchronous ??? synchronous based on XML and all xml defects
  • 6.
    What is socool? REpresentational State Transfer } Resource http://a.com/resources/ http://a.com/resources/item1/ http://a.com/resources/item2/ http://a.com/resources/item3/ Representation Representation Representation GET PUT POST DELETE THE VERBS
  • 7.
    What is socool? REpresentational State Transfer 1. Scalability 6. In HTTP context but not limited to this protocol. 2. Generalization of interface 7. Good designed interface. 3. Independence 4. Security 5. Simplicity
  • 8.
  • 9.
    Benchmark JSON responsefor frameworks and languages. Python frameworks: * Django-stripped 13 269 per second * Flask - 11 506 per sec * Django - 7122 per sec http://www.techempower.com/blog/2013/04/05/frameworks-round- 2/
  • 10.
    The source codeand problems https://github.com/TechEmpower/FrameworkBenchmarks/ http://www.techempower.com/blog/2013/04/05/frameworks-round-2/ In this souce code for flask and django isn't used connection pool - cache of database connection. connection connection database connection database connection connection for each request is connection are maintained for open and closed. future
  • 11.
    Solutions? 1. Database optimalizationfor PostgreSQL: ● pgBouncer ● pgPool http://www.askthepony.com/blog/2011/07/django-and-postgresql-improving-the-performance-with-no-effort-and-no-code/
  • 12.
    Solutions? code profiler can tell us more... 2. JSON standard serialization in Python STD library is slow, so we can improve performance by using module ujson - ultra json.
  • 13.
    Solutions? 3. Rendering templatecan be faster with template engine jinja (inspired by Django templates)
  • 14.
    Flask ● "micro" does not means micro possibilities, ● core is light, simple and extensible, ● support many extensions in each layer (for example your database layer can by relational database or non-relational data persistence).
  • 15.
    Case study -micro framework Flask from sqlalchemy import Column, Integer, String from newsletter.database import Base class Member(Base): __tablename__ = 'newsletter_members' id = Column(Integer, primary_key=True) last_name = Column(String(50)) first_name = Column(String(120)) email = Column(String(120), unique=True) def __init__(self, last_name=None, first_name=None, email=None): self.last_name = last_name self.first_name = first_name self.email = email def __repr__(self): return '<Member %r>' % (self.last_name)
  • 16.
    Case study -micro framework Flask class API(MethodView): def get(self, member_id): if member_id is None: return Member.query.all() else: return Member.query.filter_by(id = member_id).first() def post(self, data ): member = Member(first_name = data['first_name'], email=data['email']) db.session.add(member) db.session.commit() return 'OK' app.add_url_rule('/users/<int:user_id>', view_func=API.as_view('user_api'), methods=['GET', 'POST']) #class flask.views.MethodView and recognizing each REST methods are based on generic dispatch_request()
  • 17.
    Case study -micro framework Flask It was simple and quick to code? improve simplicity and spent time with... Flask RESTless
  • 18.
    Flask RESTless -full script import flask import flask.ext.sqlalchemy import flask.ext.restless app = flask.Flask(__name__) app.config['DEBUG'] = True app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db' db = flask.ext.sqlalchemy.SQLAlchemy(app) class Member(db/Model): __tablename__ = 'newsletter_members' Too simple for real project.. ? id = db.Column(Integer, primary_key=True) last_name = db.Column(String(50)) first_name = db.Column(String(120)) email = db.Column(String(120), unique=True) db.create_all() manager = flask.ext.restless.APIManager(app, flask_sqlalchemy_db=db) manager.create_api(Member, methods=['GET', 'POST']) app.run()
  • 19.
    Flask RESTless -more sugar ● versioning: GET /api/member GET /api/member/(int: id) GET /api/member?q=<json> POST /api/member apimanager.create_api(Member, url_prefix='/api/v2') GET /api/v2/member ● validation: manager.create_api(Member, methods=['GET', 'POST'], validation_exceptions= [ValidationError]) { "validation_errors": { "email": "Email is not valid..", } }
  • 20.
    Flask RESTless -more sugar ● specify columns (include or exclude): apimanager.create_api(Member, include_columns = ['last_name', 'email']) ● pagination: manager.create_api(Member, methods=['GET', 'POST'], results_per_page=2) { "validation_errors": { "age": "Must be an integer", } { "num_results": 6, "total_pages": 3, "page": 1, "objects": [ {"last_name": "Kovalsky", "email": "kovalsky@gmail.com", "id": 1}, {"last_name": "Novak", "email": "novak@gmail.com", "id": 2} ] }
  • 21.
    Flask RESTless -more and more.. ● pre/post-processors: def pre_get_single(instid): # do something pass def pre_get_many(params): # do something pass # Create an API for the Member model. manager.create_api(Person, methods=['GET', 'POST'], # A list of preprocessors for each method. preprocessors={'GET_SINGLE': [pre_get_single], 'GET_MANY': [pre_get_many],}) ● Authentication: def auth_func(params): if not current_user.is_authenticated(): raise ProcessingException(message='Not authenticated!') return NO_CHANGE manager.create_api(Person, preprocessors={'GET_SINGLE': [auth_func]})
  • 22.
    Flask RESTless -more and more.. ● filtering: import requests import json url = 'http://127.0.0.1:5000/api/member' headers = {'Content-Type': 'application/json'} filters = [dict(email='email', op='like', val='%y%')] params = dict(q=json.dumps(dict(filters=filters))) response = requests.get(url, params=params, headers=headers) GET /api/member?q={"filters":[{"name":"email", "op":"like", "val": "kovalsky"}]} OPERATORS examples: ○ ==, eq, !=, neq, ○ >, gte, <, lte FOR RELATIONS: ○ in, not_in ● column__column example: member__group ○ is_null ○ like ○ has ○ any
  • 23.
    What about Django? ● Define your model: from django.db import models class Member(models.Model): last_name = models.CharField(max_length = 100, verbose_name = "Last name") first_name = models.CharField(max_length = 100, verbose_name = "First name") email = models.EmailField(max_length = 100, verbose_name = "Email") def __unicode__(self): return self.email class Meta: verbose_name = "Newsletter member" verbose_name_plural = "Newsletter members"
  • 24.
    What about Django? ● Define your model serializer: class MemberSerializer(serializers.ModelSerializer): class Meta: model = Member fields = ('last_name', 'first_name', 'email') ● Working with serializers: from newsletter.models import Member from newsletter.serializers import MemberSerializer from rest_framework.renderers import JSONRenderer from rest_framework.parsers import JSONParser member = Member(last_name='Kovalsky', first_name= 'Johny', 'email' = 'kovalsky@gmail. com') member.save() serializer = MemberSerializer(member) content = JSONRenderer().render(serializer.data) # content: {"pk": 2, "last_name": "Kovalsky", "first_name="Johny", email = "kovalsky@gmail.com"}, more object in format [{'foo': 'bar'}, {'foo': 'bar'}]
  • 25.
    What about Django? ● Define your class-based views: class MemberDetail(APIView): """ Retrieve, update or delete a member instance. """ def get_object(self, pk): try: return Member.objects.get(pk=pk) except Member.DoesNotExist: raise Http404 def get(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member) return Response(member.data)
  • 26.
    What about Django? ● Define your class-based views: def put(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member, data=request.DATA) if serializer.is_valid(): serializer.save() return Response(serializer.data) return Response(serializer.errors, status=status. HTTP_400_BAD_REQUEST) def delete(self, request, pk, format=None): member = self.get_object(pk) member.delete() return Response(status=status.HTTP_204_NO_CONTENT)
  • 27.
    What about Django? ● Or generic views: from newsletter.models import Member from newsletter.serializers import MemberSerializer from rest_framework import generics class MemberList(generics.ListCreateAPIView): model = Member serializer_class = MemberSerializer class MemberDetail(generics.RetrieveUpdateDestroyAPIView): model = Member serializer_class = MemberSerializer ● URLs patterns: urlpatterns = patterns('', url(r'^members/$', views.MeberList.as_view()), url(r'^members/(?P<pk>[0-9]+)/$', views.MemberDetail.as_view()), )
  • 28.
    RESTful API -done http://sarahwmackey.files.wordpress.com/
  • 29.
    Thank you foryour attention http://solution4future.com/