I would say it’s not really an issue (yet) as Crypto.SE tends to be a bit less noisy and less chaotic compared to sites like (for example) StackOverflow. Nevertheless, it surely wouldn’t hurt to drop votes more frequently.
After all, it’s the votes that sum up to the user score.
If you check the voting stats, it shows that merely 10 people managed to drop 1000+ votes in here. Since some users have been around for more than a year, I can’t ignore the feeling this could be enhanced just a little bit. (I tend to see more people commenting than voting each and every day.)
In the end, I can only repeat what I said before in another answer to a somewhat related question…
…
Your vote is your voice… show members that their question or answer is a good one and that you appreciate the effort they’ve put into it.

Upvoting only takes a single click – and comes with a free, nice feeling of having shared some kudos.
As the “Winterbash 2015” is still ongoing, maybe each user reading this should ask himself/herself : Did I manage to earn a “vote early, vote often” hat this year? If not, ask yourself why not… it’s pretty easy to gain by simply casting a total of 250 votes on any seven consecutive UTC days. Besides that, it would surely help pave the way towards resolving potential issues arising from the currently rather low number of “trusted users”.
(As an aside I would like to note that I personally “trust” more users around here than their individual user scores might indicate. Too bad the system doesn’t take into account such personal opinions.)