Craig: I think some of that might depend on the location. Here in central California, the farmer's markets where I shop are small, and many of the vendors are local and do grow what they sell. I actually know some of them, because they are customers of my husband's outdoor power equipment dealership. Some vendors come in from the central valley (Sacramento and south of there, which is 2-3 hours away) and sell here, but many have farms located within an hour or so of the market. I will agree that often, the person SELLING the produce is not the farmer/farm owner, but an employee of the farm or co-op, but there are a number of tiny stands selling just a very few items (variety-wise, not quantity-wise) and usually it is the actual grower doing the selling there.
Now, in other areas I'm sure what you say is true. I just happen to live in a farming community where a lot of the food for the whole country is grown. I vividly recall my trip to Puerto Rico, and the absence of fruit stands that are so familiar in most tropical areas. When we finally saw a fruit stand and stopped, we clearly saw that the fruit being sold came from Dominican Republic and was grocery-store quality. We were told that the government regulated fruit stands and we would not find local produce sold. Huge disappointment.
