I think you're doing this the right way, and sylvievere gives good advice about
Safe Calls, but also know that they work best when you make sure your date KNOWS that someone is watching your back. The article describes some precautions that may be a little over the top and maybe not so necessary, but the concept is something I think everyone should be familiar with, and not just for kink dates, but for ALL dates. Some people even go so far as to have a trusted friend IN the coffee shop and sitting at another table, unbeknownst to their date. You know the risks and you can decide what level of precaution makes sense for you in this situation.
Let them see you talking to your safe call and let them hear you telling them where you are and what you're doing. It can also be a good opportunity to get in a good flirt. "Yeah, we're still at the coffee shop and don't have any plans yet to go anywhere. He's more handsome than his picture, and I'm having a really good time!" Also use a distinction with your safe call that sounds innocuous. For instance, if you say everything is "cool," then that means everything is fine. If you told them everything is "just fine," then that might be the agreed upon code phrase that means everything is NOT fine at all and you'd like them to come and visit you at the coffee shop NOW.
Not playing on a first date for some people is just a good rule of thumb. What's the hurry? If he would play with you tonight, but not next week, that should tell you something. I would also suggest making sure you have your own transportation to and from the date, and insist on that for future dates until you have a strong enough gut feeling about the person.