Download to read offline
Uploaded byJose Luis Martínez
Building an aws sdk for Perl - Granada Perl Workshop 2014
This document summarizes the challenges and strategies for building an AWS SDK for Perl. It discusses that while there are many Perl modules on CPAN for individual AWS services, they are inconsistent and not always up to date. Building a comprehensive SDK is challenging due to the large number of services, API versions, signatures, and AWS's rapid pace of innovation. The document outlines approaches taken, including using roles and Moose to dynamically generate classes, introspecting official SDKs, and making the SDK data-driven. It also discusses lessons learned around namespaces, enums, request objects, and calling conventions. It encourages contributors to help on the open source GitHub project.
Building an aws sdk for Perl - Granada Perl Workshop 2014
- 1. Building an AWS SDK GranadaPerl Workshop 2014 Jose Luis Martinez JLMARTIN on CPAN Twitter: @pplu_io joseluis.martinez@capside.com
- 2. About me CTO@CAPSiDE Training Partner for AWS Delivering AWS official training All four AWS certifications Real experience with real platforms Deliver consulting and continuous operation Writing Perl for +10 years And still having fun with it!
- 3. About AWS Mostlyknown for EC2 and S3 But lot's more services! (+30) Everything is an API “Programmable datacenter” Software Defined Everything
- 4. About AWS Mostlyknown for EC2 and S3 But lot's more services! (+30) Everything is an API “Programmable datacenter” Software Defined Everything It’s the your wet dream
- 5. Perl & AWS Badnews No official SDK Official ones: Ruby, PHP, Python, JS, .Net, Java Note: python wasn't official initially!
- 6. Perl & AWS Goodnews CPAN And an active community Lots of modules for different services (sometimes more than one)
- 7. Perl & AWS:The CPAN Inconsistencies In method naming Arbitrary defaults ('eu-west-1', 'us-east-1') Not up to date with latest APIs Keeping up with AWS is hard!!! “Not so popular” services not covered IAM, CloudFormation, SWF, RedShift, CloudSearch, etc Almost no Role support AWS::CLIWrapper is a good generic solution. Not native
- 8. So lets writean entire SDK!
- 10.
- 11. You can endup with… or
- 12. Challenges: # ofservices 30+ services Some with 100+ API calls Each call with it’s parameters Want to return objects Evolution
- 13. Challenges: # ofservices
- 14. So lets startclassifying
- 15. Challenges: Endpoints Endpoint TypeServices Regional Services available in a región the rest Single Global services 6
- 16. Challenges: WebService Types OneURL REST Params: Query Response: XML 17 3 Params: JSON Response: JSON 10 1
- 17. Challenges: Signatures Type Services v22 v3https 1 cloudfront 1 v4 the rest
- 18. Challenges: API versions More tan one API version is live That’s why not-up-to-date callers still work RDS 4 versions CloudFront 3 versions EC2 4 versions
- 19. AWS Pace ofinnovation Every month there are changes Changes mean APIs change (slightly) Very hard to keep up!
- 20. AWS Pace ofinnovation Every week there are announcements Changes mean APIs change (slightly) Very hard to keep up!
- 21.
- 22. Hand write classesand methods, and parameters
- 23. Hand write classesand methods, and parameters
- 24. Parse docs Afterall… you would have to read them to write the classes
- 25. Parse docs Afterall… you would have to read them to write the classes
- 26. No published spec Introspect an official SDK! After all AWS already has read the docs for you
- 27. No published spec Introspect an official SDK! That’s your spec! After all AWS already has read the docs for you
- 28. Data driven SDKs! JS SDK has datastructures that define calls Proof of concept Win++! Boto and PHP2 too! Later migrate to using Boto
- 29. Data driven SDKs! JS SDK has datastructures that define calls Proof of concept Win++! Boto and PHP2 too! Later migrate to using Boto
- 30.
- 31. Modern Perl tothe rescue Try to be very lazy Parameter validation: Create Moose classes that validate the parameters passed to the call. May change in the future. Got very quick results! Hide all possible implementation from the user Try not to leak implementation That way you can change it at will Meta lets you inspect the objects! Call objects get marshalled into datastructured via meta Result datastructures get marshalled into objects via meta
- 32.
- 33. Modern Perl: Roles All functionality gets pulled in via Roles Code is basically in different roles, Classes are “serialized configurations” of how to call APIs Makes it easy to Generate service classes from datastructures Change behaviour for tests
- 34. Roles are usedfor One global endpoint vs endpoints per region Caller (Query, JSON) Response (XML, JSON) Signatures (V2, V3, V4) IO
- 35. Aws is aclass factory
- 36. Dynamic class construction User never constructs service classes directly Aws class method uses default config my $ec2 = Aws->service(‘EC2’)->new( . . . )
- 37. Dynamic class construction User wants custom functionality my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::TestCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) service loads Aws::EC2, creates a new class with TestCaller
- 38. Win: Hello tothe async world Still playing around in examples my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::MojoCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) my $asgs = $ec2->DescribeAutoScalingGroups; See examples/asyncAutoScaling.pl
- 39. Win: Also going“fully async” my $aws = Aws->new( config => AWS::SDK::Config->new(caller => 'Net::AWS::MojoAsyncCaller') ); my $delay = Mojo::IOLoop->delay(sub { my ($delay, @responses) = @_; Dumper($_) for @responses; }); foreach my $region ( "us-east-1", "ap-northeast-1", "sa-east-1", "ap-southeast-1", "ap-southeast-2", "us-west-2", "us-west-1", "eu-west-1", 'invented-region') { my $ctrail = $aws->service('CloudTrail')->new( region => $region, delay => $delay, ); print "Doing a call for $regionn"; my $list = $ctrail->DescribeTrails; } $delay->wait;
- 41.
- 42. Lessons learned Firstshot with MooseX::Declare Error messages are HORRIBLE! Change to Moops didn't go well Finally plain old Moose
- 43. Lessons learned Namespacefinding (still in the works) AWS vs Aws Thanks @clintongormley for the article!
- 44. Lessons learned Enums(still in the works) APIs sometimes return unexpected stuff
- 45. Lessons learned Abstract$request object passed around Finally marshalled into the UserAgents’ taste
- 46. Still open stuff Load time for big services (EC2) Not all classes will be used all the time Generate classes on demand Calling convention Now CamelCased Not so perlish Calling from objects (not service objects) $queue->DeleteQueue(); from a queue object Attribute Traits Access to Arrays and HashRefs via nicer methods Lots more!!!
- 47. Use AWS? UsePerl? Want to help? Fork your heart out: https://github.com/pplu/aws-sdk-perl/