Debugging, Troubleshooting Distributed Web & Cloud Applications San Diego, July 27th, 2013 SolCal Code Camp
Theo Jungeblut • Engineering manager & lead by day at AppDynamics in San Francisco • Coder & software craftsman by night • Architects decoupled solutions tailored to business needs & crafts maintainable code to last • Worked in healthcare and factory automation, building mission critical applications, framework & platforms • Degree in Software Engineering and Network Communications • Enjoys cycling, running and eating theo@designitright.net www.designitright.net
Warning • Contains also proprietary software • This areas are pointed out • Please walk away now, if that is not acceptable for you, it’s okay
Overview • Inquiry about the Status Quo • Some Facts -> Dev Process & Ops Reality • Monolithic Applications • Debugging & Logging • Distributed Applications • Side Effects you need to troubleshot • Performance Counter • Microsoft IntelliTrace • AppDynamics • Summary • Q & A
We need YOU!
How agile are You?
The Impact of Process * from Wikipedia: http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg
How many releases do your company have a year?
Typical Releases a Year* 58% 17% 21% 3% 2+ months monthly weekly daily http://www.appdynamics.com/blog/2011/12/14/storm-clouds-in-2012-summary-of- appdynamics-apm-customer-survey/
Development + Operations = DevOps * from Wikipedia: http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg
Difference between Dev & Ops Dev • Getting / being agile: need for frequent updates with new feature and bug fixes • Achieves stability trough running latest Ops • Highest goal: stable execution flawless execution, by knowing the environment and software • Achieves stability through careful rollouts
“The Past/Present”* Web Server DBUser * or maybe still your present?
“The Past/Present” - Architecture Data Access Layer Business Logic Layer Presentation Layer DB
How are you troubleshooting?
Traditional Debugging
Traditional Debugging Advantages • Visibility for executed code path Disadvantages • Performance impact • Requires software installation • Often not possible in production • Typical does not find easily distributes issues • Only real time results
Log Files Advantages • Output adjustable without restart • Accepted in Production Disadvantages • Manual, time intense creation of log statements • Manual, time intense process of result review • Limited scope & visibility • Potentially high performance impact based on IO
“The Present/Future”- Architecture Microsoft MSDN: http://msdn.microsoft.com/es-es/architecture/gg189193
“The Present/Future” ASP.NET MVC 3/4 (IIS) User WAS Tomcat Azure Web Role SQL Server MySQL Active Directory Azure Worker Role WAS 3rd Party Web Services Java Application MongoDB 3rd Party Web Services WCF HTTP
The (Netflix) Distributed Reality * * http://techblog.netflix.com/2012/06/netflix-operations-part-i-going.html
What has this to do with Debugging and Troubleshooting?
Who does really know how their application is performing?
What impacts Application Performance?
Concurrency http://en.wikipedia.org/wiki/File:Smithposter.jpg • Deadlocks • Raise Conditions • Unexpected complexity • …
Data Volume http://en.wikipedia.org/wiki/File:Smithposter.jpg • constantly increasing • Volume hides details • Handling volume impacts resources • …
Resources http://blog.laptopmag.com/amd-cries-foul-but-tests-show-battery-life-inferior-to-intels/battery_morpheus • are limited • Danger of Starvation • …
Who is Performance Testing every release?
http://www.slideshare.net/appdynamics/what-can-devops-learn-from-formula-1
Dev + Ops = Success • DevOps has to be a combined approach • Mostly no visibility in real performance & production state
IntelliTrace™ • “Flight Recorder” for your Process • Records configurable details • Allows offline analysis • Microsoft specific Tool in Visual Studio Let’s take a look http://msdn.microsoft.com/en-us/library/hh398365.aspx
Summary IntelliTrace Advantages • Allows great visibility into single process/AppPool • Allows to play back test or production issue Disadvantages • Limited to Single Process / AppPool Visibility • Should be only temporally enable in production • Requires configuration effort for balance between details & performance
Performance Counter • Collect machine wide and process specific data • Allows on- and offline analysis • Microsoft specific tool build into Windows http://www.codeproject.com/Articles/8590/An-Introduction-To-Performance-Counters
Summary Performance Counter Advantages • Huge OOTB support in Windows/IIS/etc. • Low overhead • Extensible Disadvantages • Collecting application specific data requires source code changes
AppDynamics • Application Performance Management • Optimized for Production • Monitoring instead of investigating • Providing the “Right-Data” Let’s take a look http://www.appdynamics.com/products-editions.php
Summary AppDynamics Advantages • Works in production on high performance sides • Monitors, Notify, Troubleshoot • Allows instance handling in cloud environments • Free lite version Disadvantages • Full version not free • Does not include every application detail
Build your own Opinion (for Free) • Lite Version free but limited • 30 days free Trial of Pro & Azure Version http://www.appdynamics.com/products-free-download.php
Summary • Understand Dev Impact to Ops Execution • Understand the business impact of production issues • Monolithic is simpler to Troubleshoot • Distributed is more complex • The right tools can make a Big Difference
Downloads, Feedback & Comments: Q & A Graphic by Nathan Sawaya courtesy of brickartist.com theo@designitright.net www.designitright.net www.speakerrate.com/theoj www. slideshare.net/theojungeblut
Blog, Rating, Slides http://www.DesignItRight.net www.speakerrate.com/theoj www.slideshare.net/theojungeblut
… thanks for you attention! And visit and support the www.sandiegodotnet.com Please fill out the feedback, and… www.speakerrate.com/theoj

Debugging,Troubleshooting & Monitoring Distributed Web & Cloud Applications at SoCal Code Camp San Diego (07/27/2013)