Can you exemplify some of the situations where you actually encounter non-shortest forms, besides the Java encoding which has been discussed here? Since you feel so strongly about allowing them, I take it you get them frequently?
I otoh have the distinct impression that they don't occur except in exploits (except, again, the Java NUL trick). To me it's sort of like not fixing stack smashing bugs for fear of incompatibilities.