As with rolling Attributes, rolling for Damage is something that can seriously bog down a game. I recommend doing the same sort of short-cut creation here as you would there; figure out what the averages, and just use that instead of rolling when dealing with insignificant NPCs. Save the rolling of the dice for when it's someone that matters dealing with someone else that matters; "mook rules" are popular for a reason.
The scheme is the same: figure the average of a given roll of the dice, use that as a static number instead, and adjust it if you see a need to show what someone more or less proficient than average would do. The catch here is that you can play with some dice schemes (like 3d4x10) to get a better sense of what the average would be. (e.g. 3d4x10 is better averaged as 6d4x5; 2d4 averages to 5, multipled by 3 is 15, and then by 5 is 75)
If you go so far as to make static numbers of insignificant NPCs' Attack and Defense rolls (which you do by adding 10 to their relevant bonuses, noting the auto-fail for a natural 1 and auto-hit for a natural 20) so that the players do all of the dice-rolling, then you've automated NPCs to the point where you can focus on just the rolls of significant characters (the PCs and those NPCs important enough to merit individual attention) and thereby devote your attention accordingly.
Mull it over. This won't work for all of you, but for some of you this will be a great house rule to adopt. I shift depending on what I want out of the gameplay experience, and so--I advise--should you.