Support mark_dynamic

[JackCaoG]

This pr is to support the use case of

compiled = torch.compile(mode) torch._dynamo.makr_dynamic(input, 0) res = compiled(input) 

where the 0th dimension of input can be dynamic. I fixed a couple things in this pr

1. Currently we remove the symints from the args but filter on the tensor type, This is actually problematic because the FX graph actually expects symints to be part of the input. This will likely incur the edge cases where the order/number of args are different from what fx expects

2. We currently cache each graph by all input shapes. I took a profile and this can be expensive(~1ms for 16 layer decoder). Instead pytorch actually pass us the dynamic dimension as input so we can cache on that instead. If there is no dynamic dimension we will cache on an empty tuple which is cheap.

3. with dynamic=False and mark_dynami we can no longer easily tell if current graph will be dynamic or not. Since with the 2 above, the caching will be very cheap, so I do the graph_var lookup for every graph instead of dynamic graph only

4. We can not run the partitioner when there are symints as inputs because partitioner will put the symint and its return into a separate graph which mess up the dynamic dimension look up. For now I just skip partitioner when there is an symint inputs.

[JackCaoG]

@ysiraichi FYI, I made the assumption that input to dynamo is either an int or a tensor. Let me know if this is not a correct assumption.

[wonjoo-wj]

🚀