We have a horror story to relate to you.. in regards to this decision
We met a couple a couple years younger than us, and we had enjoyed a few evenings of fun.. and had progress to hanging out for regular fun, having a dinner or BBQ..They knew we had kids and flat out refused to do anything with the kids in the area.. even flirting.. We knew they had a very young daughter, and again out of respect flat out refused to do anything, with children in the vicinity.
We had a "date" set for a later evening we arrived at 10 pm, and progressed a bit slowly, the clothes all came off, and as we swapped players.. The wife and I had moved into a cowgirl and he and my wife were engaging a bit of doggy,a few feet away, when out of the bedroom comes thier little girl, crying.. She jumps off me, and her father was trying to keep going with my wife.. which she pulled away from..
As they were calming thier little girl, we dressed and as they came back out expecting to continue we said good nite.. When we got home, now its like 1am.. there was a email from them, at first apologising in the first paragraph then it turned a bit nasty as they were ranting about why we left..
I hope this helps you in your decision, personally, we were offended that we had been lied to, knowing our position on the subject..
And lets put a finer point on it, Even behind locked doors, there arises moments were a child NEEDS Mommy or Daddy (nightmares, Dirty Diapers, Drinks, simple comfort).. lets face it.. In the heat of ANY moment.. thats a Buzzkill.. know what I mean.. Be a parent first