Air Vent dirt or Rust?

I have a 1996 Honda Prelude.
I bought it used a little over 2 years ago. I have noticed a slow accumulation of brown dust in the air vent pockets. Specifically
where the air vent connects to the drivers front door. (you see the vent when you open the door, it is in the dash. ).. But I noticed it in other places as well.
I have cleaned it before (vacuum), but it always comes back.
I thought it might be an interior vent air filter deteriorating, but the service manual doesn't mention one.
The particles are bigger than dust particles, and brown in color. A friend said he had something similar with a home A/C, where it turned out to be rust.
I rarely use the A/C, and usually use the heater, and/or the blower is on a lot of the time, blowing unconditioned air much of the time.
I live in a fairly dry (low humidity) climate, northern california..
I suppose I will need to remove the blower/heater unit and inspect it, but I am wondering if anyone has seen this before?
Randall Fox

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
Add image file
Upload is a website by car enthusiasts for car enthusiasts. It is not affiliated with any of the car or spare part manufacturers or car dealers discussed here. All logos and trade names are the property of their respective owners.