I have kids (they are 7 and 8 now)....but even when they were toddlers, I would not have considered having playmates over at the house. And as swingers now, we as a couple are not comfortable with the idea of playing with children in the house, no matter what their age.
I'm sure there are some potential playmates out there that would have absolutely no problem coming over.
Now, the question I am most curious about...many parents try to find a reliable/trustworthy baby sitter in pretty short order just for having an evening out now and again. Have you had problems with past babysitters or is there just some preconceived notion that wanting to have a night out to yourselves to do adult things makes you horrible parents? (Hey, I used to be married to one of those...and at best we went out 2-4 times per year...Valentines Day, anniversary, and a company christmas party were guaranteed 'date nights'.)
And as far as the video monitor in the barn. Handy if you are out there just doing work or puttering around...probably not so ideal during a playtime. To effectively monitor the monitor, it means one of you is probably not paying attention to your play partner.
Good luck!