I noticed heavy use of LINQ and 'ToArray' which is generating a lot of garbage per frame. Could be avoided by just omitting the LINQ and allocating lists and clearing/reusing them every frame instead of generating new arrays each time.