I'd thought about doing something like that once upon a time, but never did.
I suspect that the fine cloud of filth/bacteria that will erupt from the gravel will be rather fine and not entirely trapped by the filter, leading to a tank that is cloudy for a period after doing this.
... but maybe I'm wrong and if you do it, let us know how it turns out!
