Compare OCI Runtimes

Transcript

1. ͘͞ΒΠϯλʔωοτגࣜձࣾ (C) Copyright 1996-2019 SAKURA Internet Inc ͘͞ΒΠϯλʔωοτݚڀॴ OCIϥϯλΠϜൺֱͷͨΊʹ ΍͍ͬͯΔ͜ͱ͋Ε͜Ε

   2019/03/22 ٬һݚڀһ ٶԼ ߶ี runc, gVisor, Kata Containers Nabla Containers, Firecrackerൺֱ

2. 1. ͸͡Ίʹ

3. 3 ɾݱࡏOCIϥϯλΠϜͷൺֱʹऔΓ૊ΜͰ͍Δ ɾൺֱ߲໨͸ɺىಈ࣌ؒɺϝϞϦαΠζɺύϑΥʔϚϯεͳͲ ɾϥϯλΠϜຖʹบ͕͋Γɺಉ͡৚݅Ͱൺֱ͢Δͷ͕؆୯Ͱ͸ͳ͍ ɾ۩ମతʹ΍ͬͨ͜ͱɺͱ͘ʹ͸·ͬͨ͜ͱڞ༗͍ͨ͠ ɾܭଌ݁Ռͷڞ༗͸͜ͷൃදͷ໨తͰ͸ͳ͍ ɾݱࡏͷܭଌ݁Ռʹ͍ͭͯ͸matsumotory͞ΜͷεϥΠυࢀর ɾhttps://speakerdeck.com/matsumoto_r/chao-ge-ti-xing- detasentaostoocirantaimu ͓࿩͢͠Δ಺༰

4. 1. ܭଌ४උͰ͸·ͬͨ͜ͱ

5. ίϯςφͰಈ͔͢όΠφϦ࡞੒

6. 6 ɾൺֱ৚݅Λἧ͑ΔͷͱɺϥϯλΠϜͦͷ΋ͷͷੑ࣭Λଌఆ͍ͨ͠ͷ Ͱɺ୯७ͳϓϩάϥϜΛ༻ҙ ɾhello.c ɾHelloͱදࣔ͢Δ͚ͩͷϓϩάϥϜ ɾىಈ࣌ؒଌఆ༻ ɾloop.c ɾແݶϧʔϓ͢ΔϓϩάϥϜ ɾϝϞϦαΠζଌఆ༻ ίϯςφͰಈ͔͢όΠφϦ

7. 7 hello.c #include <stdio.h> void main() { printf("Hello\n"); }

8. 8 loop.c #include <stdio.h> void main() { int i =

   0; while(1) { printf("%d\n", i++); } }

9. 9 ɾNabla Containers͸ϥΠϒϥϦOSʢϢχΧʔωϧʣܕΞϓϩʔνͷ ϥϯλΠϜ ɾ࣮ߦόΠφϦͱΧʔωϧ͕ҰମԽ ɾͳͷͰNabla Containers༻όΠφϦ͸ผʹ༻ҙ͢Δඞཁ͕͋Δ ɾx86_64-rumprun-netbsd-gcc -o hello.out

   hello.c ɾrumprun-bake solo5_ukvm_seccomp hello.nabla hello.out ɾ৽͠໨ͷϦϏδϣϯͩͱsolo5_ukvm_seccompͰ͸ͳ͘spt Nabla Containers༻όΠφϦ

10. 10 ɾnabla-containers/solo5ΛίϯύΠϧͯ͠Ͱ͖ͨsolo5.oΛ/usr/lib/ libsolo5_seccomp.aʹίϐʔ ɾnabla-containers/runnc ͸ϦϏδϣϯb78fe29Λར༻ ɾnabla-containers/rumprun͸ϦϏδϣϯ8b01b3Λར༻ ɾߋʹҎԼͷύονΛ͋ͯΔ ɾhttps://github.com/rumpkernel/rumprun/issues/122 ɾhttps://github.com/rumpkernel/rumprun/pull/118 Nabla༻όΠφϦͷϏϧυ

11. 11 ɾrumprun-bakeίϚϯυʹ΋ύον ɾ࠷ޙʹ࣮ߦͯ͠ΔίϚϯυʹ-L/usr/libΛ௥Ճ ɾhttps://blog.cloudkernels.net/posts/build-a-nabla-docker-image/ ɾ͜ͷखॱʹ͕ͨͬͯ͠࠷৽ͷϦϏδϣϯͰϏϧυͯ͠΋͏·͘ಈ͔ ͳ͔ͬͨ ɾSolo5: ABORT: spt/net.c:36: Assertion

    `netfd >= 0' failed Nabla༻όΠφϦͷϏϧυʢ͖ͭͮʣ

12. Kata Containersͷόʔδϣϯ

13. 13 ɾhttps://github.com/kata-containers/documentation/blob/master/ install/ubuntu-installation-guide.md ɾܭଌ༻ϗετʹUbuntuΛར༻ͨ͠ͷͰ͜ͷखॱʹैͬͨ ɾhttp://download.opensuse.org/repositories/home:/ katacontainers:/releases:/${ARCH}:/master/xUbuntu_$ (lsb_release -rs)/ ͕aptϦϙδτϦͱͯ͠ઃఆ͞ΕΔ ɾ͕ɺ͜Εͩͱ1.6rc1͕Πϯετʔϧ͞Ε·ͱ΋ʹಈ͔ͳ͔ͬͨ

    ɾs/master/stable-1.5/ Ͱղܾ Kata ContainersͷΠϯετʔϧ

14. 1. ܭଌ࣌ʹ͸·ͬͨ͜ͱ

15. ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ

16. 16 ɾ༨෼ͳϨΠϠʔΛল͍ͯͳΔ΂͘ૉͷঢ়ଶͰͷܭଌ͕໨త ɾOCI Filesystem BundleΛ༻ҙ ɾdocker export `docker create mizzy/hello`

    | tar -C bundle/rootfs - xvf - Ͱrootfsੜ੒ ɾrunc specͰconfig.jsonੜ੒ ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ

17. 17 ɾrunc, gVisor, Kata Containers, Nabla Containersʹ͸OCI Filesystem BundleΛ࣮ߦ͢ΔίϚϯυ͕͋Δ ɾFirecrackerʹ͸(·ͩ?)ଘࡏ͠ͳ͍ͬΆ͍

    ɾͳͷͰFirecracker͸௚઀࣮ߦํࣜͰ͸ܭଌͰ͖ͳ͔ͬͨ ɾkata-fc࢖͑͹Ͱ͖ͦ͏ʢະணखʣ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support OCI Filesystem Bundle࣮ߦίϚϯυ

18. 18 ɾrunnc͸ଞͷίϚϯυͱҧ͍runαϒίϚϯυ͕ͳ͍ ɾcreateͯ͠start͢Δඞཁ͕͋Δ ɾtime runnc startͰܭଌ͠Α͏ͱ͢Δͱίϯςφ࣮ߦऴྃલʹtime ͷ݁Ռ͕ฦΔ → ྑ͍ܭଌํ๏໛ࡧத ɾconfig.jsonͷhooks.prestartͰωοτϫʔΫ·ΘΓͷઃఆΛߦ͏ඞ

    ཁ͕͋Δ ɾhttps://github.com/nabla-containers/runnc/issues/53 ɾconfig.jsonͰࢦఆ͢Δroot.path͕૬ରύεͩͱಈ͔ͳ͍ Nabla Containersׂ͕ͱۂऀ

19. containerdͷctrίϚϯυͰͷܭଌ

20. 20 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰ͸͢΂ͯͷϥϯλΠϜΛܭଌͰ͖ͳ ͔ͬͨͷͰҧ͏ΞϓϩʔνͰܭଌ ɾ͜͜Ͱ΋Nabla Containersͷนཱ͕ͪ͸͔ͩΔ ɾଞͷϥϯλΠϜ͸Shim API v2ʹରԠ͍ͯ͠Δ ɾctr run

    —runtime=io.containerd.kata.v2 Έ͍ͨʹ࣮ߦͰ͖Δ ɾrunnc͸Shim API v2ʹରԠ͍ͯ͠ͳ͍ containerdͷctrίϚϯυͰͷܭଌ

21. 21 ɾ/etc/containerd/config.toml ɾctr run --runtime io.containerd.runtime.v1.linux Ͱ࣮ߦ ผͷํ๏Ͱ࣮ߦΛࢼΈΔ [plugins] [plugins.linux]

    shim = "containerd-shim" runtime = "/usr/local/bin/runnc"

22. 22 ɾctr: OCI runtime create failed: runnc did not terminate

    sucessfully: unknown ɾrunnc͕panic: Insufficient uniqueness in IDΛు͍ͯΔ ɾཁ͢Δʹίϯςφ໊͕୹͍ ɾϩάʹ͸͜Ε͕ݟ͋ͨΒͳ͍ͷͰΘ͔Γʹ͍͘ ɾERR: could not create tapabcdefg12345: no master interface: Link not found ɾίϯςφ໊Λ௕͘͢Δͱࠓ౓͸͜ͷΤϥʔ ɾ͜ΕҎ্͸·ͩௐࠪͰ͖͍ͯͳ͍ ࣮ߦ݁Ռ

23. dockerίϚϯυͰͷܭଌ

24. 24 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌɺctrίϚϯυͰͷܭଌɺͱ΋ ʹ͢΂ͯͷϥϯλΠϜΛܭଌ͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ ɾͷͰ࣍͸dockerίϚϯυͰτϥΠ dockerίϚϯυͰͷܭଌ

25. 25 ɾFirecrackerͷಈ͔͠ํ͚ͩΘ͔ΒΜɺͱࢥͬͨΒudzura͞ΜʹΑΔ φΠεࢿྉ͕ ɾhttps://speakerdeck.com/udzura/firecracker-from-low-layer-to- hight?slide=14 ɾKata ContainersͰFirecrackerΛಈ͔͢kata-fcΛར༻ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support ɾDockerͷdevicemapperαϙʔτ͕ඞཁ͕ͩɺݱࡏ࠷৽ͷ18.09͕

    devicemapperରԠ͍ͯ͠ͳ͍ͷͰɺ18.06Λར༻͢Δඞཁ͋Γ Docker + Firecraker

26. 26 ɾ࠷ॳ͸ҙຯ͕Θ͔Βͳ͔ͬͨ ɾ͑ɺͲͬͪ΋OCIϥϯλΠϜ͡Όͳ͍ͷʁ ɾFiracracker͸ϚΫϩͳࢹ఺ͰݟΔͱOCIϥϯλΠϜͱͯ͠ݟΔ͜ͱ ΋Ͱ͖Δ͕ϛΫϩͳࢹ఺ͰݟΔͱVMM ɾKata Containers͸VM಺ͰίϯςφΛىಈ͢ΔΞϓϩʔνͷOCIϥ ϯλΠϜ ɾVMMͱͯ͠σϑΥϧτͰQEMUΛར༻͢Δ͕ࠩ͠ସ͑Մೳ ɾͭ·ΓKata

    ConͷVMMΛFirecrackerʹࠩ͠ସ͑Δ͜ͱ͕Ͱ͖Δ Kata Containers + Firecracker?

27. ctrͱdockerͰFirecrackerͷ ىಈ͕࣌ؒ૝ఆͱҟͳΔ

28. 28 ɾctr: real 0m6.320s ɾdocker: real 0m4.105s ɾdockerͷํ͕ɺdockerdΛܦ༝͢Δ෼஗͘ͳΓͦ͏ͳͷʹͳͥʁ ɾctr͸naive snapshotterΛར༻

    ɾdocker͸devicemapperΛར༻ ɾctrͰdevmapper snapshotterΛར༻͢Ε͹ಉ͡৚݅ͰൺֱͰ͖ͦ͏ ɾ→ ະணख ctrͱdockerͰͷFirecrackerىಈ࣌ؒ

29. 1. ίετ

30. 30 ɾݕূ؀ڥΛVagrant+VirtualBoxͰߏங ɾKataͱFirecracker͸KVM͕ඞཁ ɾVirtualBoxͰ͸KVMಈ͔ͳ͍ ɾVMWare Fusion + Vagrant VMWare ProviderΛߪೖ

    ɾVMWare Fusion: 9,925ԁ ɾVagrant VMWare Provider: $79 per seat ׂͱ͓͕͔͔ۚΔ (on macOS)

31. 31 ɾVagrant + VMWare FusionͰmodprobe vhost_vsock͕Τϥʔʹ ͳͬͯ͠·͏ͷͰAWS EC2্Ͱ΋ݕূ ɾKVMΛಈ͔ͨ͢ΊʹϕΞϝλϧΠϯελϯε͕ඞཁ ɾi3.metalͰ4.992USD/࣌ؒ

    ɾ1೔ͰBilling AlertඈΜͰདྷͨ ׂͱ͓͕͔͔ۚΔ (on AWS)

32. 1. ࢀߟࢿྉ

33. 33 ɾhttps://github.com/mizzy/container-playground ɾmeasurements/ ɾVagrant + VMWare FusionͰಈ͔ͯ͠Δ΍ͭ ɾcompare_on_i3_metal/ ɾVagrant +

    AWS EC2 i3.metalΠϯελϯεͰಈ͔ͯ͠Δ΍ͭ ɾ੔ཧͰ͖ͯͳ͍͠ɺ௨͠Ͱvagrant provisionͯ͠ͳ͍ͷͰಈ͔ͳ ͍ͱ͜Ζ͋Γͦ͏ ɾࢼͯ͠ΈͯΘ͔Βͳ͍͜ͱ͕͋Ε͹ԿͰ΋ฉ͍͍ͯͩ͘͞ ܭଌ༻ϦϙδτϦ

34. 34 ɾࠓ࿩୊ͷ͍Ζ͍ΖͳίϯςφϥϯλΠϜΛൺֱͯ͠Έͨ ɾhttps://www.slideshare.net/KoheiTokunaga/ss-123664087 ɾ֤छϥϯλΠϜͷಛ௃΍ൺֱͳͲͱͯ΋ࢀߟʹͳΔ ɾNabla ContainersΛಈ͔͢ʹ͋ͨͬͯͱͯ΋ࢀߟʹͳͬͨ ɾࢿྉͰ͸ܭଌʹkubernetes-sigs/cri-toolsΛར༻͍ͯ͠ΔͷͰ͜Ε ΋ࢼͯ͠Έ͍ͨ ࢀߟࢿྉ