Code Forensics Diego Pacheco
@diego_pacheco ❏ Cat's Father ❏ Head of Software Architecture ❏ Agile Coach ❏ SOA/Microservices Expert ❏ DevOps Practitioner ❏ Speaker ❏ Author diegopacheco http://diego-pacheco.blogspot.com.br/ About me... https://diegopacheco.github.io/tinyurl.com/diegopacheco
Things we will talk about... Things you should avoid. DON'TS01 Science 101 How to run a investigation02 Observability, Debug Tricks, Linux Tools... Tools & Tricks03 Post Mortems, RCAs and Retrospectives Closing the Case04
Don'ts
Don't: FREAK OUT ❏ Avoid extra pressure on yourself ❏ Avoid worry about time. ❏ Avoid making comparisons like this should be done in 1h or so. ❏ As time pass is easy to Freak Out ❏ Making progress is your friend, don't nail it down only about solving the mystery. ❏ Your eyes can trick you, use tools. ❏ Make sure you do things properly. ❏ Comparing Strings ❏ Read Errors Carefully. ❏ Don't forget to Breath
Don't: Do two many things at time. ❏ Don't change 2 things at the time. ❏ Code ❏ Vars / Config ❏ Trys ❏ One Hypotesy at the time ❏ Otherwise how do you know what did what? ❏ You need to be: ❏ Methodical ❏ Boring ❏ Slow
How to Run a investigation Science 101 Have Theories Write Down Facts Have a Partner (Pair Programing) Minimize Investigation Efforts
Science 101
Science 101: Do we have a smoking gun?
Minimize Investigation Efforts 4% of code == 72% of defects (Code as Crime Scene)
Have Theories ❏ It's a like a guessing game, how the murder happen? ❏ Think on most likely thing it could be. ❏ There are classical Offenders like: ❏ NPE Lack of validation/test ❏ Code Change ❏ Config Change ❏ Credentials Change ❏ Typo somewhere ❏ Lack of Security groups ❏ Not enough Resources (memory, space) ❏ OOM Killer
Write down Facts ❏ Write down (paper, txt file, evernote, google docs) ❏ Write Important fact like: ❏ Class Names, Methods, Variables ❏ Make sure you don't get lots. ❏ It's easy to forget important pieces of information when you are worried and want to fix the issue fast.
Have a Partner (Pair Programing)
Have a Partner (Pair Programing)
Have a Partner (Pair Programing)
Tools & Tricks ❏ Observability ❏ Debug tricky's ❏ Linux Tools ❏ Write your own tools
Observability
—Filipe Fortes "Debugging is like being the detective in a crime movie where you are also the murderer." -
Debug tricks ❏ Know your IDE. ❏ Remote Debugging. ❏ Debug tricks like: ❏ Conditionals ❏ Log Expressions ❏ Runtime eval ❏ Exception Breakpoints ❏ Field Break points
Linux Tools
Write your own tools ❏ Help to nail down problems faster. ❏ Simple Utilitary. ❏ Program or scripts (Groovy, Python , Rust, Go). ❏ Quick diagnostics.
Could you compare with something else? ❏ It's very likely the issue is a code change, so maybe was working before. Some digging is needed. ❏ Not always possible (i.g new feature) ❏ Use diff tools. ❏ Compare previous versions that worked. ❏ Run previous tests that worked. ❏ Debug and write down differences.
Closing the Case Fix, Share & Improve ❏ Adding More Tests ❏ Post Mortems ❏ RCAs ❏ Retrospectives
Adding more Tests ❏ Boys Scout rules. ❏ Closing the Case means you found the issue. ❏ So we need to create test to simulate the issue. ❏ Having the test will proof we don't suffer again.
Closing the Case: Post Mortems
Closing the Case: RCA ❏ Similar to Post Mortems. ❏ Lean tool. Can be used with: 5 whys, fishbone and other system thinking tools. ❏ Excel file: ❏ Issue ❏ classification ❏ Why it happen ❏ How we make sure it does not happen again
Closing the Case: Retrospectives
Good Investigation Books
Code Forensics Diego Pacheco

Code Forensics

  • 1.
  • 2.
    @diego_pacheco ❏ Cat's Father ❏Head of Software Architecture ❏ Agile Coach ❏ SOA/Microservices Expert ❏ DevOps Practitioner ❏ Speaker ❏ Author diegopacheco http://diego-pacheco.blogspot.com.br/ About me... https://diegopacheco.github.io/tinyurl.com/diegopacheco
  • 3.
    Things we will talkabout... Things you should avoid. DON'TS01 Science 101 How to run a investigation02 Observability, Debug Tricks, Linux Tools... Tools & Tricks03 Post Mortems, RCAs and Retrospectives Closing the Case04
  • 4.
  • 5.
    Don't: FREAK OUT ❏Avoid extra pressure on yourself ❏ Avoid worry about time. ❏ Avoid making comparisons like this should be done in 1h or so. ❏ As time pass is easy to Freak Out ❏ Making progress is your friend, don't nail it down only about solving the mystery. ❏ Your eyes can trick you, use tools. ❏ Make sure you do things properly. ❏ Comparing Strings ❏ Read Errors Carefully. ❏ Don't forget to Breath
  • 6.
    Don't: Do twomany things at time. ❏ Don't change 2 things at the time. ❏ Code ❏ Vars / Config ❏ Trys ❏ One Hypotesy at the time ❏ Otherwise how do you know what did what? ❏ You need to be: ❏ Methodical ❏ Boring ❏ Slow
  • 7.
    How to Runa investigation Science 101 Have Theories Write Down Facts Have a Partner (Pair Programing) Minimize Investigation Efforts
  • 8.
  • 9.
    Science 101: Dowe have a smoking gun?
  • 10.
    Minimize Investigation Efforts 4%of code == 72% of defects (Code as Crime Scene)
  • 11.
    Have Theories ❏ It'sa like a guessing game, how the murder happen? ❏ Think on most likely thing it could be. ❏ There are classical Offenders like: ❏ NPE Lack of validation/test ❏ Code Change ❏ Config Change ❏ Credentials Change ❏ Typo somewhere ❏ Lack of Security groups ❏ Not enough Resources (memory, space) ❏ OOM Killer
  • 12.
    Write down Facts ❏Write down (paper, txt file, evernote, google docs) ❏ Write Important fact like: ❏ Class Names, Methods, Variables ❏ Make sure you don't get lots. ❏ It's easy to forget important pieces of information when you are worried and want to fix the issue fast.
  • 13.
    Have a Partner(Pair Programing)
  • 14.
    Have a Partner(Pair Programing)
  • 15.
    Have a Partner(Pair Programing)
  • 16.
    Tools & Tricks ❏Observability ❏ Debug tricky's ❏ Linux Tools ❏ Write your own tools
  • 17.
  • 18.
    —Filipe Fortes "Debugging islike being the detective in a crime movie where you are also the murderer." -
  • 19.
    Debug tricks ❏ Knowyour IDE. ❏ Remote Debugging. ❏ Debug tricks like: ❏ Conditionals ❏ Log Expressions ❏ Runtime eval ❏ Exception Breakpoints ❏ Field Break points
  • 20.
  • 21.
    Write your owntools ❏ Help to nail down problems faster. ❏ Simple Utilitary. ❏ Program or scripts (Groovy, Python , Rust, Go). ❏ Quick diagnostics.
  • 22.
    Could you comparewith something else? ❏ It's very likely the issue is a code change, so maybe was working before. Some digging is needed. ❏ Not always possible (i.g new feature) ❏ Use diff tools. ❏ Compare previous versions that worked. ❏ Run previous tests that worked. ❏ Debug and write down differences.
  • 23.
    Closing the Case Fix,Share & Improve ❏ Adding More Tests ❏ Post Mortems ❏ RCAs ❏ Retrospectives
  • 24.
    Adding more Tests ❏Boys Scout rules. ❏ Closing the Case means you found the issue. ❏ So we need to create test to simulate the issue. ❏ Having the test will proof we don't suffer again.
  • 25.
    Closing the Case:Post Mortems
  • 26.
    Closing the Case:RCA ❏ Similar to Post Mortems. ❏ Lean tool. Can be used with: 5 whys, fishbone and other system thinking tools. ❏ Excel file: ❏ Issue ❏ classification ❏ Why it happen ❏ How we make sure it does not happen again
  • 27.
    Closing the Case:Retrospectives
  • 28.
  • 29.