Downloaded 46 times
![Demo App Note: Used HashSets, Arrays[][], Lists, and Vectors](https://image.slidesharecdn.com/exploringheapdumpsv1-181023032517/75/Exploring-Java-Heap-Dumps-Oracle-Code-One-2018-57-2048.jpg)
Uploaded byRyan Cuprak
Exploring Java Heap Dumps (Oracle Code One 2018)
The document discusses Java heap dumps, highlighting their role in memory analysis, specifically addressing issues such as memory leaks and excessive garbage collection in applications. It also explores the tools and techniques for generating, analyzing, and interpreting heap dumps, including the use of the NetBeans profiler and Object Query Language (OQL). Best practices for heap analysis are provided, emphasizing the importance of managing memory and identifying data model errors effectively.
Exploring Java Heap Dumps (Oracle Code One 2018)
- 1.
- 2. Java Heap Review •Java objects are stored in the heap • All objects are globally reachable in the heap • Heap is created when an application starts • Size of heap is configured using –Xmx and –Xmx • Garbage collection prunes the heap and removes objects no longer reachable • Stack memory - variable values are stored when their methods are invoked Heap Contains Everything and can be DUMPED to DISK
- 3. Why Analyze Heaps? •IT reports Java EE/Spring server memory footprint has grown to 9 gigs • Server app logs contain OutOfMemoryExceptions • Connections to queueing or database are exhausted • Serialized Java objects in queue are unreasonably large • Desktop application becomes unresponsive • Excessive amount of garbage collection
- 4. Java Heap Analysis MAT Profiler Allyou need is a profiler right?
- 5. Memory Leaks Textbook memory leaks- easy to find and fix. JConsole
- 6. Production Heap Dumps •9 gigs of data • 13k classes loaded • ~ 136 million instances • ~6,000 GC roots
- 7. Production Heap Dumps Capturethe Heap Dump and…
- 8.
- 9. Heap Dump Panic Toomuch data! • Impossible to comprehend • No human way to explore the data • Application data model is too complicated
- 10. Real Memory Leaks BankAccount: 1231209 Owner Bob Owner JulieReport January 2018 Bank Account: 1231210 Bank Account: 1231209 Report January 2018 Owner Bob Challenge: Data looks good everywhere…
- 11. Real Memory Leaks Causes: •Faulty clone methods • Duplicate singletons • Accidently cached data • Cache logic bugs Complications • May NOT GROW over time (leaks gets cleaned-up) • More than one non-trivial memory leak
- 12. What about OQL? •OQL • Object Query Language – used for querying heaps • SQL-like language • Supports JavaScript expressions • Supported in NetBeans and VisualVM • Downside • Poorly documented and hard to use • Easy to create runaway queries
- 13.
- 14. NetBeans Profiler • NetBeansis open source IDE/platform • Modular architecture • Clean code base Profiler GUI Profiler API
- 15. NetBeans Profiler API •Parses hprof files • Creates an object model representing the hprof file • Pages data in from disk • Simple API (master in about 10 minutes) • Independent of NetBeans • Can be extract and use in any IDE – Plain old Java Talk is really about how to build a custom heap analysis tool: • To answer specific data model questions • With custom logic for your data model
- 16.
- 17. Generating Heap Dumps •Command line parameter: • -XX +HeapDumpOnOutOfMemoryError • Command line: • jmap –dump:format=b,file=dump.hprof <pid> • jhsdb jmap --binaryheap --pid <pid> • jcmd <pid> GC.heap_dump <file name> • Ctrl-Break Command Line
- 18.
- 19.
- 20. Heap Dump Warning Dumpingthe heap: • Takes time • Consumes diskspace • Negatively affects performance
- 21. Targeted Heap Dumps •Serialize object graphs from application to a file. • Read the serialized data into another tool and then programmatically create a heap dump.
- 22.
- 23. Building Custom Profiler CreateNetBeans Platform App Copy API src out of NetBeans
- 24.
- 25. NetBeans Platform App Adddependency on “Java Profiler (JFluid)”
- 26. Profiler Sources Checkout source: https://github.com/apache/incubator-netbeans.git Profilercode: netbeans/profiler/lib.profiler/src/netbeans/lib.profiler/heap Copy heap directory
- 27. Which Approach? • Copyingsources easiest • Most analysis apps are command line (one-offs) - Note - You don’t need the classpath of the application from which the heap was generated.
- 28.
- 29.
- 30. Heap Object Methods getJavaClassByName(Stringfqn) : JavaClass getAllClasses() : List getBiggestObjectsByRetainedSize(int number) : List getGCRoots(): GCRoot getInstanceByID(long instanceId) : Instance getJavaClassByID(long javaclassId) : JavaClass getJavaClassesByRegExp(String regexp) : HeapSummary getSummary() : Properties
- 31.
- 32. Heap Summary • getTotalLiveInstances(): long • getTime() : long • getTotalAllocatedBytes() : long • getTotalAllocatedInstances() : long • getTotalLiveBytes() : long
- 33. Exploration Starting Points •GCRoots • Threads (really GCRoots) • Class Types
- 34. GC Roots • GarbageCollection Root is an object that is accessible from outside the heap. • Objects that aren’t accessible from a GC Root are garbage collected • GC root categorization: • Class loaded by system class loader • Thread • Stack Local • Monitor • JNI Reference • Held by JVM
- 35. Garbage Collection Roots Javaframe: 44 thread object: 5 JNI global: 29 sticky class: 1284
- 36.
- 37. GC Roots JNI_GLOBAL ="JNI global"; JNI_LOCAL = "JNI local"; JAVA_FRAME = "Java frame"; NATIVE_STACK = "native stack"; STICKY_CLASS = "sticky class"; THREAD_BLOCK = "thread block"; MONITOR_USED = "monitor used"; THREAD_OBJECT = "thread object"; UNKNOWN = "unknown"; INTERNED_STRING = "interned string"; FINALIZING = "finalizing"; DEBUGGER = "debugger"; REFERENCE_CLEANUP = "reference cleanup"; VM_INTERNAL = "VM internal"; JNI_MONITOR = "JNI monitor"; root.getKind() : String
- 38. Finding Classes • Canperform lookup using: • Fully qualified class name (ex. java.lang.String) • Class ID • Instance ID • IDs are unique to heap dump • Hash codes are not available!
- 39. Profiler Data Model JavaClass InstanceB Value Value Instance A Value Value
- 40.
- 41. Instances From an instance: •Who references the instance • Who does the instance reference Perform instanceof to find out: • ObjectArray • PrimitiveArray GCRoot can take forever…
- 42. Values If you askan instance for its references, you get a list of Value objects.
- 43. Example: Member Variables Iteratesover all Person objects and prints member variables.
- 44.
- 45.
- 46.
- 47. String Extraction Strings areobjects – array of characters
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54. Noise: Ignore InternalClasses Ignore internal JVM classes
- 55. Puzzler Prints: Count: 231 •230 entries have 1 are referenced by one other object • 1 entry is “owned” by 822 other objects
- 56.
- 57. Demo App Note: UsedHashSets, Arrays[][], Lists, and Vectors
- 58.
- 59.
- 60.
- 61. String Utilization • 5159Strings in heap dump • 15 associated with data model
- 62.
- 63. Data Model Leak Addlogic to fire an employee:
- 64. Data Model Leak FiredAdam – shouldn’t be in the system!
- 65.
- 66.
- 67. Best Practices • Bemindful of your heap • Cache analysis on disk when processing large heaps • Heap processing is I/O bound • Not all profiler calls are the same • Look for Javadoc: Speed: normal • Maintain a list of processed objects • Easy to run in circles • Exclude JVM internal classes from analysis • Revisit graph algorithms!
- 68. Summary • Heap snapshotcan be easily explored • Excellent way to verify application logic • Only way to identify deep data model/logic errors • Can be used to recover data • Generate a heap snapshot from a frozen/corrupted application and then mine
- 69. Q&A Twitter: @ctjava Email: rcuprak@gmail.com/ r5k@3ds.com Blog: cuprak.info Linkedin: www.linkedin.com/in/rcuprak Slides: www.slideshare.net/rcuprak/presentations
Editor's Notes
- #20 APM tools