This might be seen as just unnecessary for optimizing binaries, but you'd have to at least ask yourself, why does .NET do what leads to these solutions?
For historical reasons,
System.Stringuses the UCS-2 character encoding, that is, UTF-16 without surrogate pairs.
However, most strings in typical .NET applications consist solely of ASCII characters, leading to wasted space: half of the bytes in a string are likely to be null bytes!
Since strings are immutable, we can scan the character data when the string is constructed, then dynamically select an encoding, thereby saving 50% of string memory in most cases.
For some reason, this applies to all string literals except enum names and field names.
The fork linked in this page hasn't been active since 2016, and I know, before I even try, if I clone it and run make that there will be a million errors. So now, it's time to pull out my best gimmick in programming, hacks.
Tests
Here is a base executable with no code to execute:
3.0KB:
0000: 4D5A90000300000004000000FFFF0000 MZ.............. 0010: B8000000000000004000000000000000 ........@....... 0020: 00000000000000000000000000000000 ................ 0030: 00000000000000000000000080000000 ................ 0040: 0E1FBA0E00B409CD21B8014CCD215468 ........!..L.!Th 0050: 69732070726F6772616D2063616E6E6F is program canno 0060: 742062652072756E20696E20444F5320 t be run in DOS 0070: 6D6F64652E0D0D0A2400000000000000 mode....$....... 0080: 504500004C0103000000000000000000 PE..L........... 0090: 00000000E00002010B01080000040000 ................ 00a0: 0006000000000000BE22000000200000 ........."... .. 00b0: 00400000000040000020000000020000 .@....@.. ...... 00c0: 04000000000000000400000000000000 ................ 00d0: 00800000000200000000000003004085 ..............@. 00e0: 00001000001000000000100000100000 ................ 00f0: 00000000100000000000000000000000 ................ 0100: 702200004B00000000400000D0020000 p"..K....@...... 0110: 00000000000000000000000000000000 ................ 0120: 006000000C0000000000000000000000 .`.............. 0130: 00000000000000000000000000000000 ................ 0140: 00000000000000000000000000000000 ................ 0150: 00000000000000000020000008000000 ......... ...... 0160: 00000000000000000820000048000000 ......... ..H... 0170: 00000000000000002E74657874000000 .........text... 0180: C4020000002000000004000000020000 ..... .......... 0190: 00000000000000000000000020000060 ............ ..` 01a0: 2E72737263000000D002000000400000 .rsrc........@.. 01b0: 00040000000600000000000000000000 ................ 01c0: 00000000400000402E72656C6F630000 ....@..@.reloc.. 01d0: 0C0000000060000000020000000A0000 .....`.......... 01e0: 00000000000000000000000040000042 ............@..B 01f0: 00000000000000000000000000000000 ................ 0200: A0220000000000004800000002000500 ."......H....... 0210: 5C2000000C0200000100000002000006 \ .............. 0220: 00000000000000000000000000000000 ................ 0230: 00000000000000000000000000000000 ................ 0240: 00000000000000000000000000000000 ................ 0250: 1E02280100000A2A062A000042534A42 ..(....*.*..BSJB 0260: 01000100000000000C00000076322E30 ............v2.0 0270: 2E35303732370000000005006C000000 .50727......l... 0280: D0000000237E00003C01000084000000 ....#~..<....... 0290: 23537472696E677300000000C0010000 #Strings........ 02a0: 0800000023555300C801000010000000 ....#US......... 02b0: 2347554944000000D801000034000000 #GUID.......4... 02c0: 23426C6F620000000000000002000010 #Blob........... 02d0: 471500000900000000FA013300160000 G..........3.... 02e0: 01000000020000000200000002000000 ................ 02f0: 01000000020000000100000001000000 ................ 0300: 0100000000007B000100000000000600 ......{......... 0310: 17001E00060034005200000000000100 ......4.R....... 0320: 0000000001000100000010000A000000 ................ 0330: 05000100010050200000000086182500 ......P ......%. 0340: 0100010058200000000096002B000500 ....X ......+... 0350: 01000000010012000900250001001100 ..........%..... 0360: 250001002E0013000B00048000000000 %............... 0370: 00000000000000000000000030000000 ............0... 0380: 0200000000000000000000002A007200 ............*.r. 0390: 0000000000000000003C4D6F64756C65 .........<Module 03a0: 3E0050726F6772616D0061726773004F >.Program.args.O 03b0: 626A6563740053797374656D002E6374 bject.System..ct 03c0: 6F72004D61696E007768790052756E74 or.Main.why.Runt 03d0: 696D65436F6D7061746962696C697479 imeCompatibility 03e0: 4174747269627574650053797374656D Attribute.System 03f0: 2E52756E74696D652E436F6D70696C65 .Runtime.Compile 0400: 725365727669636573006D73636F726C rServices.mscorl 0410: 6962007768792E657865000000032000 ib.why.exe.... . 0420: 000000001B8EF60B53E43C4E97C8F435 ........S.<N...5 0430: 7DB1B9050003200001050001011D0E1E }..... ......... 0440: 01000100540216577261704E6F6E4578 ....T..WrapNonEx 0450: 63657074696F6E5468726F77730108B7 ceptionThrows... 0460: 7A5C561934E089000000000000000000 z\V.4........... 0470: 982200000000000000000000AE220000 ."...........".. 0480: 00200000000000000000000000000000 . .............. 0490: 0000000000000000A022000000000000 ........."...... 04a0: 00005F436F724578654D61696E006D73 .._CorExeMain.ms 04b0: 636F7265652E646C6C0000000000FF25 coree.dll......% 04c0: 00204000000000000000000000000000 . @............. 04d0: 00000000000000000000000000000000 ................ At the bare minimum, executables have an extra 1.5KB of data for the .reloc and .rsrc section, which the latter has version info.
DEV is not helping me here with the little width I have to work with.
Plain String
Here's a plain call to WriteLine in Program.Main.
Console.WriteLine("Lorem ipsum dolor sit amet, consectetur... Compile command: mcs -debug- -nostdlib- -o+ -sdk:2 test.cs
Binary view: (xxd -u -g 1 -s +0x200 -l 0x400 test.exe)
4.0KB:
03a0: 2F0084000000000000000000003C4D6F /............<Mo 03b0: 64756C653E0050726F6772616D006172 dule>.Program.ar 03c0: 677300436F6E736F6C65005379737465 gs.Console.Syste 03d0: 6D0057726974654C696E65004F626A65 m.WriteLine.Obje 03e0: 6374002E63746F72004D61696E007768 ct..ctor.Main.wh 03f0: 790052756E74696D65436F6D70617469 y.RuntimeCompati 0400: 62696C69747941747472696275746500 bilityAttribute. 0410: 53797374656D2E52756E74696D652E43 System.Runtime.C 0420: 6F6D70696C6572536572766963657300 ompilerServices. 0430: 6D73636F726C6962007768792E657865 mscorlib.why.exe 0440: 000000000084054C006F00720065006D .......L.o.r.e.m 0450: 00200069007000730075006D00200064 . .i.p.s.u.m. .d 0460: 006F006C006F00720020007300690074 .o.l.o.r. .s.i.t 0470: 00200061006D00650074002C00200063 . .a.m.e.t.,. .c 0480: 006F006E007300650063007400650074 .o.n.s.e.c.t.e.t 0490: 00750072002000610064006900700069 .u.r. .a.d.i.p.i 04a0: 007300630069006E006700200065006C .s.c.i.n.g. .e.l 04b0: 00690074002E0020004E0075006E0063 .i.t... .N.u.n.c 04c0: 00200065006700650074002000610075 . .e.g.e.t. .a.u 04d0: 00630074006F0072002000740065006C .c.t.o.r. .t.e.l 04e0: 006C00750073002C0020007500740020 .l.u.s.,. .u.t. 04f0: 0063006F006E00640069006D0065006E .c.o.n.d.i.m.e.n 0500: 00740075006D0020006500730074002E .t.u.m. .e.s.t.. 0510: 0020004E0075006C006C006100200066 . .N.u.l.l.a. .f 0520: 00650072006D0065006E00740075006D .e.r.m.e.n.t.u.m 0530: 002C0020007400750072007000690073 .,. .t.u.r.p.i.s 0540: 002000730069007400200061006D0065 . .s.i.t. .a.m.e 0550: 0074002000680065006E006400720065 .t. .h.e.n.d.r.e 0560: 00720069007400200072007500740072 .r.i.t. .r.u.t.r 0570: 0075006D002C00200061006E00740065 .u.m.,. .a.n.t.e 0580: 00200064007500690020006C006F0062 . .d.u.i. .l.o.b 0590: 006F0072007400690073002000610075 .o.r.t.i.s. .a.u 05a0: 006700750065002C0020007300650064 .g.u.e.,. .s.e.d 05b0: 002000700075006C00760069006E0061 . .p.u.l.v.i.n.a 05c0: 007200200065007800200064006F006C .r. .e.x. .d.o.l 05d0: 006F00720020006E006F006E0020006C .o.r. .n.o.n. .l 05e0: 0061006300750073002E0020004E0061 .a.c.u.s... .N.a 05f0: 006D0020007300650064002000650073 .m. .s.e.d. .e.s This string in binary takes the size of an EXE section. (0x400 bytes / 1KB)
This is doomsday for large console apps.
String Resource
Loading a string from a resource:
Console.WriteLine(Resources.ResourceManager.GetString("test")); Resource file:
<?xml version="1.0" encoding="utf-8"?> <root> ... <data name="test" xml:space="preserve"> <value>Lorem ipsum dolor sit amet, consectetur...</value> </data> </root> Command: resgen /useSourcePath /compile main.resx
Added to MCS args: -resource:main.resources
4.5KB:
0200: D0280000000000004800000002000500 .(......H....... 0210: 80230000200500000100000004000006 .#.. ........... 0220: AC200000D20200000000000000000000 . .............. 0230: 00000000000000000000000000000000 ................ 0240: 00000000000000000000000000000000 ................ 0250: 1E02280900000A2AD27E010000041428 ..(....*.~.....( 0260: 0500000A391E0000007201000070D002 ....9....r...p.. 0270: 000002280600000A6F0700000A730800 ...(....o....s.. 0280: 000A80010000047E010000042A1E0228 .......~....*..( 0290: 0900000A2A5628020000067209000070 ....*V(....r...p 02a0: 6F0A00000A280B00000A2A00CE020000 o....(....*..... 02b0: CECAEFBE01000000910000006C537973 ............lSys 02c0: 74656D2E5265736F75726365732E5265 tem.Resources.Re 02d0: 736F757263655265616465722C206D73 sourceReader, ms 02e0: 636F726C69622C2056657273696F6E3D corlib, Version= 02f0: 342E302E302E302C2043756C74757265 4.0.0.0, Culture 0300: 3D6E65757472616C2C205075626C6963 =neutral, Public 0310: 4B6579546F6B656E3D62373761356335 KeyToken=b77a5c5 0320: 3631393334653038392353797374656D 61934e089#System 0330: 2E5265736F75726365732E52756E7469 .Resources.Runti 0340: 6D655265736F75726365536574020000 meResourceSet... 0350: 00010000000000000050414450414450 .........PADPADP 0360: 33AF737C00000000C900000008740065 3.s|.........t.e 0370: 0073007400000000000182044C6F7265 .s.t........Lore 0380: 6D20697073756D20646F6C6F72207369 m ipsum dolor si 0390: 7420616D65742C20636F6E7365637465 t amet, consecte 03a0: 7475722061646970697363696E672065 tur adipiscing e 03b0: 6C69742E204E756E6320656765742061 lit. Nunc eget a 03c0: 7563746F722074656C6C75732C207574 uctor tellus, ut 03d0: 20636F6E64696D656E74756D20657374 condimentum est 07a0: 0000000000000000B700230100000000 ..........#..... 07b0: 0000000001000000E401000000000000 ................ 07c0: 003C4D6F64756C653E00520050726F67 .<Module>.R.Prog 07d0: 72616D00724D005265736F757263654D ram.rM.ResourceM 07e0: 616E616765720053797374656D2E5265 anager.System.Re 07f0: 736F75726365730047656E6572617465 sources.Generate 0800: 64436F64654174747269627574650053 dCodeAttribute.S 0810: 797374656D2E436F6465446F6D2E436F ystem.CodeDom.Co 0820: 6D70696C6572002E63746F7200446562 mpiler..ctor.Deb 0830: 75676765724E6F6E55736572436F6465 uggerNonUserCode 0840: 4174747269627574650053797374656D Attribute.System 0850: 2E446961676E6F737469637300436F6D .Diagnostics.Com 0860: 70696C657247656E6572617465644174 pilerGeneratedAt 0870: 747269627574650053797374656D2E52 tribute.System.R 0880: 756E74696D652E436F6D70696C657253 untime.CompilerS 0890: 6572766963657300456469746F724272 ervices.EditorBr 08a0: 6F777361626C65417474726962757465 owsableAttribute 08b0: 0053797374656D2E436F6D706F6E656E .System.Componen 08c0: 744D6F64656C00456469746F7242726F tModel.EditorBro 08d0: 777361626C655374617465004F626A65 wsableState.Obje 08e0: 63740053797374656D00526566657265 ct.System.Refere 08f0: 6E6365457175616C7300547970650047 nceEquals.Type.G 0900: 65745479706546726F6D48616E646C65 etTypeFromHandle 0910: 0052756E74696D655479706548616E64 .RuntimeTypeHand 0920: 6C65006765745F417373656D626C7900 le.get_Assembly. 0930: 417373656D626C790053797374656D2E Assembly.System. 0940: 5265666C656374696F6E006172677300 Reflection.args. 0950: 476574537472696E6700436F6E736F6C GetString.Consol 0960: 650057726974654C696E65006765745F e.WriteLine.get_ 0970: 4D004D61696E004D007768790052756E M.Main.M.why.Run 0980: 74696D65436F6D7061746962696C6974 timeCompatibilit 0990: 79417474726962757465006D73636F72 yAttribute.mscor 09a0: 6C6962007768792E7265736F75726365 lib.why.resource 09b0: 73007768792E65786500000000077700 s.why.exe.....w. 09c0: 68007900000974006500730074000000 h.y...t.e.s.t... 09d0: 7855E192F3A8DF4ABD3B8148F7C608B2 xU.....J.;.H.... 09e0: 0003061205052002010E0E4101003353 ...... ....A..3S 09f0: 797374656D2E5265736F75726365732E ystem.Resources. 0a00: 546F6F6C732E5374726F6E676C795479 Tools.StronglyTy 0a10: 7065645265736F757263654275696C64 pedResourceBuild 0a20: 65720831372E302E302E300000032000 er.17.0.0.0... . This adds nearly as much bloat with the bottom data.
String as Byte[]
(Real world example)
(sigh):
Console.WriteLine(Encoding.ASCII.GetString( new byte[] {(byte)'L',(byte)'o',(byte)'r',(byte)'e',(byte)'m',(byte)' ',(byte)'i'...} )); 5.0KB (...):
0100: F02300004B00000000600000D0020000 .#..K....`...... 0110: 00000000000000000000000000000000 ................ 0120: 008000000C0000000000000000000000 ................ 0130: 00000000000000000000000000000000 ................ 0140: 00000000000000000000000000000000 ................ 0150: 00000000000000000020000008000000 ......... ...... 0160: 00000000000000000820000048000000 ......... ..H... 0170: 00000000000000002E74657874000000 .........text... 0180: 44040000002000000006000000040000 D.... .......... 0190: 00000000000000000000000020000060 ............ ..` 01a0: 2E736461746100000402000000400000 .sdata.......@.. 01b0: 00040000000A00000000000000000000 ................ 01c0: 00000000400000C02E72737263000000 ....@....rsrc... 01d0: D00200000060000000040000000E0000 .....`.......... 01e0: 00000000000000000000000040000040 ............@..@ 01f0: 2E72656C6F6300000C00000000800000 .reloc.......... 0200: 00020000001200000000000000000000 ................ 0630: 03000000003C4D6F64756C653E005072 .....<Module>.Pr 0640: 6F6772616D0061726773004279746500 ogram.args.Byte. 0650: 53797374656D003C5072697661746549 System.<PrivateI 0660: 6D706C656D656E746174696F6E446574 mplementationDet 0670: 61696C733E0024417272617954797065 ails>.$ArrayType 0680: 3D35313600246669656C642D31323333 =516.$field-1233 0690: 38454237413930433932343238313343 8EB7A90C9242813C 06a0: 41463436453833353941303343304242 AF46E8359A03C0BB 06b0: 444133440052756E74696D6548656C70 DA3D.RuntimeHelp 06c0: 6572730053797374656D2E52756E7469 ers.System.Runti 06d0: 6D652E436F6D70696C65725365727669 me.CompilerServi 06e0: 63657300496E697469616C697A654172 ces.InitializeAr 06f0: 7261790041727261790052756E74696D ray.Array.Runtim 0700: 654669656C6448616E646C6500436F6E eFieldHandle.Con 0710: 736F6C650057726974654C696E65004F sole.WriteLine.O 0a00: 4C6F72656D20697073756D20646F6C6F Lorem ipsum dolo 0a10: 722073697420616D65742C20636F6E73 r sit amet, cons 0a20: 65637465747572206164697069736369 ectetur adipisci 0a30: 6E6720656C69742E204E756E63206567 ng elit. Nunc eg 0a40: 657420617563746F722074656C6C7573 et auctor tellus This adds an extra section named ".sdata" where the byte array is stored, and a stupidly long field name for the array.
Enum
Probably the stupidest way to work around this, but could only work in certain circumstances like storing/using config keys.
(Real world example
Rough version that doesn't use a proper array or support punctuation:
public enum _ { Lorem, ipsum, dolor, sit, amet, consectetur, ... (dupes have underscore appended :( ) } string test = ""; for (int i = 0; i < (int)(_._); i++) test += ((_)i).ToString() + ' '; Console.WriteLine(test); 5.0KB:
03a0: 09000100010002010000120000001500 ................ 03b0: 0100030006061400010056801C000400 ..........V..... 03c0: 56802200040056802800040056802E00 V."...V.(...V... 03d0: 04005680320004005680370004005680 ..V.2...V.7...V. 03e0: 4300040056804E000400568053000400 C...V.N...V.S... 03f0: 56805800040056805D00040056806400 V.X...V.]...V.d. 0400: 040056806B00040056806E0004005680 ..V.k...V.n...V. 0410: 7A00040056807E000400568084000400 z...V.~...V..... 0420: 56808E00040056809500040056809A00 V.....V.....V... 0430: 04005680A00004005680AA0004005680 ..V.....V.....V. 0440: B10004005680B60004005680BA000400 ....V.....V..... 0450: 5680C30004005680C90004005680CD00 V.....V.....V... 0770: 5201080014015701080018015C010800 R.....W.....\... 0780: 1C016101080020016601080024016B01 ..a... .f...$.k. 0790: 08002801700108002C01750108003001 ..(.p...,.u...0. 07a0: 7A01080034017F010800380184010800 z...4.....8..... 07b0: 3C018901080040018E01080044019301 <.....@.....D... 07c0: 2E003300BC01B5010480000000000000 ..3............. 07d0: 00000000000000000000780200000200 ..........x..... 07e0: 00000000000000000000DB01BA020000 ................ 07f0: 0000030002000000003C4D6F64756C65 .........<Module 0800: 3E0050726F6772616D005F0076616C75 >.Program._.valu 0810: 655F5F004C6F72656D00697073756D00 e__.Lorem.ipsum. 0820: 646F6C6F720073697400616D65740063 dolor.sit.amet.c 0830: 6F6E7365637465747572006164697069 onsectetur.adipi 0840: 7363696E6700656C6974004E756E6300 scing.elit.Nunc. 0850: 6567657400617563746F720074656C6C eget.auctor.tell 0860: 757300757400636F6E64696D656E7475 us.ut.condimentu 0870: 6D00657374004E756C6C61006665726D m.est.Nulla.ferm 0880: 656E74756D0074757270697300736974 entum.turpis.sit This takes up 0x440 bytes.
Raw ASCII string squished into UTF-16 LE C# String
and then converted to a byte array with Unicode GetBytes,
and then converted to a string with ASCII GetString.
Console.WriteLine(Encoding.ASCII.GetString(Encoding.Unicode.GetBytes( "潌敲灩畳潤潬楳⁴浡瑥潣獮捥整畴摡灩獩楣杮攠..." ))); 3.5KB(!!):
03a0: 750017002100750017002E003B002100 u...!.u.....;.!. 03b0: 04800000000000000000000000000000 ................ 03c0: 00008000000002000000000000000000 ................ 03d0: 00004000C200000000000000003C4D6F ..@..........<Mo 03e0: 64756C653E0050726F6772616D006172 dule>.Program.ar 03f0: 677300456E636F64696E670053797374 gs.Encoding.Syst 0400: 656D2E54657874006765745F41534349 em.Text.get_ASCI 0410: 49006765745F556E69636F6465004765 I.get_Unicode.Ge 0420: 74427974657300476574537472696E67 tBytes.GetString 0430: 00436F6E736F6C650053797374656D00 .Console.System. 0440: 57726974654C696E65004F626A656374 WriteLine.Object 0450: 002E63746F72004D61696E0077687900 ..ctor.Main.why. 0460: 52756E74696D65436F6D706174696269 RuntimeCompatibi 0470: 6C697479417474726962757465005379 lityAttribute.Sy 0480: 7374656D2E52756E74696D652E436F6D stem.Runtime.Com 0490: 70696C65725365727669636573006D73 pilerServices.ms 04a0: 636F726C6962007768792E6578650000 corlib.why.exe.. 04b0: 0082034C6F72656D20697073756D2064 ...Lorem ipsum d 04c0: 6F6C6F722073697420616D65742C2063 olor sit amet, c 04d0: 6F6E7365637465747572206164697069 onsectetur adipi 04e0: 7363696E6720656C69742E204E756E63 scing elit. Nunc 04f0: 206567657420617563746F722074656C eget auctor tel 0500: 6C75732C20757420636F6E64696D656E lus, ut condimen 0510: 74756D206573742E204E756C6C612066 tum est. Nulla f 0520: 65726D656E74756D2C20747572706973 ermentum, turpis 0690: 612C20657420626962656E64756D2066 a, et bibendum f 06a0: 656C69732074656D7075732061636375 elis tempus accu 06b0: 6D73616E2E0100003EF604CCBDC0D640 msan....>......@ 06c0: B4394801D5D5B2C90004000012050520 .9H............ 06d0: 011D050E0520010E1D05040001010E03 ..... .......... 06e0: 200001050001011D0E1E010001005402 .............T. 06f0: 16577261704E6F6E457863657074696F .WrapNonExceptio 0700: 6E5468726F77730108B77A5C561934E0 nThrows...z\V.4. 0710: 89000000000000000000000000000000 ................ 0720: 4825000000000000000000005E250000 H%..........^%.. 0730: 00200000000000000000000000000000 . .............. 0740: 00000000000000005025000000000000 ........P%...... 0750: 00005F436F724578654D61696E006D73 .._CorExeMain.ms 0760: 636F7265652E646C6C0000000000FF25 coree.dll......% 0770: 00204000000000000000000000000000 . @............. 0780: 00000000000000000000000000000000 ................ The string takes 0x200 bytes / 0.5 KB.
Top comments (0)