why are sellers to be trusted, but buyers are not?

Moto_man is right, the issues are precisely why one has an escrow arrangement --very common for far more at stake. If there is ultimately disagreement the agreement provides for interpleader- the property goes to court and the parties litigate; the agent follows the court's direction. The escrow agent is usually held harmless by all (and presumably has insurance). 

I think it is a great idea, and just mentioned it elsewhere. I’ve had occasion to deal with this in all kinds of property transactions for myself. You can never be too careful, but the escrow system solves a lot of problems. Would I want to take it on as the escrow agent? No. But, done at the right scale, with the right people, it could be good for us citizens. 

The problem is that sellers expect buyers to purchase without recourse (e.g. Friends and Family).  The risk is all on the buyer.  Businesses (e.g., TMR) allow purchase by credit card giving the buyer some protection.  I dont know what the answer is beyond an escrow service but many sellers dont accept that.  

Years ago I sold an integrated amp to an Agon member who let me know that he was really hesitent to do the transaction as a few years earler he bought an item and when the carton arrived, the box was the one the factory used to ship that item when it was new, but the carton was filled with phone books...and that it took him several months to get his money refunded.  I have been fortunate with all my sale and purchase transactions, but I guess if one does enough a troubled transaction will be in their future.

Example - buyer scams seller here on Agon thread