Uploaded byChris Saint-Amant
Netflix JavaScript Talks - Scaling A/B Testing on Netflix.com with Node.js
The document discusses A/B testing strategies at Netflix, highlighting the execution of numerous tests to drive data-driven product development. It emphasizes the challenges of conditional dependencies and the benefits of templating and packaging in their development process. The final thoughts encourage rapid learning and innovation through iterative testing.
Netflix JavaScript Talks - Scaling A/B Testing on Netflix.com with Node.js
- 1. Scaling A/B testing onNetflix.com with
- 2.
- 3.
- 9.
- 10.
- 11. Test 1 A B C D E F G Test 2 A B C D E F G Test3 A B C D E F G Test 4 A B C D E F G Test 5 A B C D E F G Test 6 A B C D E F G Test 7 A B C D E F G
- 12. Test 1 A B C D E F G Test 2 A B C D E F G Test3 A B C D E F G Test 4 A B C D E F G Test 5 A B C D E F G Test 6 A B C D E F G Test 7 A B C D E F G
- 13.
- 14. hundreds of new A/Btests per year
- 15.
- 16. 2,105 566 685 CSSJSTemplates
- 17. 2.5 million unique css/jspackages per push cycle
- 18.
- 19.
- 21. ➔ Iterate Quickly ➔Complex Concepts, Minimal Code ➔ Give the Power to the People Why Node?
- 22.
- 23.
- 25. Payment Test Testing the additionof Direct Debit
- 26. Payment Test Testing the additionof Direct Debit
- 27. Payment Test Testing the additionof Direct Debit
- 28. Payment Test Testing the additionof Direct Debit
- 29. Payment Test Testing the additionof Direct Debit
- 31. payment template Control Cell2 Cell 3 Cell 4 Cell 5 if if ifif if
- 32. payment templatepayment template ControlCell 2 Cell 3 Cell 4 Cell 5 if if ifif if
- 34. payment template Control Cell2 Cell 3 Cell 4 Cell 5 if if ifif if payment_method_cc payment_method_dd
- 35. payment templatepayment template ControlCell 2 Cell 3 Cell 4 Cell 5 if if ifif if payment_method_cc payment_method_dd
- 36. payment template Control Cell2 Cell 3 Cell 4 Cell 5 payment_method_cc payment_method_dd if if if if if
- 37. payment template Control Cell2 Cell 3 Cell 4 Cell 5 payment_method_cc payment_method_dd if if if if if
- 39. payment template Control Cell 2Cell 3 Cell 4 Cell 5 payment_method_cc payment_method_dd ?
- 40. payment template Control Cell 2Cell 3 Cell 4 Cell 5 payment_method_cc payment_method_dd payment_method.json
- 41. payment template Control Cell 2Cell 3 Cell 4 Cell 5 payment_method_cc payment_method_dd payment_method.json
- 45.
- 46.
- 47.
- 48.
- 49. ➔ Developers cancombine rules ➔ Improves template legibility ➔ Increases template reuse Benefits
- 50.
- 51.
- 52.
- 55. app.js oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep
- 56. app.js oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep
- 57. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 58.
- 59. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 60. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 61.
- 62. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 63. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 64. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 65. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 66. oldSearch newSearch dep1 dep2dep3 dep4 dep5 dep6 sub-dep sub-dep sub-dep sub-dep sub-dep sub-dep sub-depsub-dep app.js
- 67.
- 69.
- 70.
- 71.
- 72.
- 73. anatomy of arule
- 74.
- 75.
- 76.
- 77.
- 79.
- 80.
- 82. Step 1: Getthe full dependency tree for the requested package from the registry.
- 83. Step 2: Runthe rules.
- 84. Step 3: Filterout all deps that resolved false.
- 85. Step 3: Filterout all deps that resolved false.
- 86. Step 3: Filterout all deps that resolved false. ✓
- 87. ✓ Step 4: Filterout all extraneous sub-deps.
- 88. Step 5: Concatenatethe files.
- 89.
- 90.
- 91. ➔ Leverage buildtime tools ➔ Leverage the server when you can ➔ Divide and conquer with modules Take aways
- 92.
- 93.
- 94. be creative with theregistry
- 95.
- 96.
- 97.
- 98. and now, the coolestpart...
- 100.
- 101.
- 102. UI BundleUI Bundle UIBundle UI Bundle UI BundleUI Bundle UI BundleUI Bundle UI Bundle
- 104.
- 105. we never touch thefile system
- 106.
- 107. ➔ Static analysisFTW ➔ Independent UI deployments ➔ Requests never touch the file system ➔ Fast package response times Wins
- 108.
- 109.
- 110.
- 111. I have notfailed. I’ve just found 10,000 ways that won’t work.” Thomas Edison “
- 112.
- 113.
- 114.