For some reason you can't sweep those here until level 50.
I never realized this until you pointed it out just now.
It took me maybe 3 weeks before I realized I can sweep Dojo. When I realized that, I was baffled by how I haven't noticed it so far.
i guess what I'm saying is, I can confirm this.