Pytorch uses different kernels depending on the input size. There is a reason why it's so massive to download.

Sure, some degree of customization is expected. However, I doubt that PyTorch implements every input size separately.

Generally tiling to a handful of sizes gets you pretty close to "specialized to this size" performance.

Sure, but the size can be anything.

Yes, you pick the size that’s closest