I think there is a fine line between treating this as some occasional fun, and an unhealthy addiction. The moment that you borrow against a line of credit, credit card or other to fund a visit, or if your mind goes in circles and you become irritable with friends and family for lack of seeing a girl, it is probably time for some soul searching or a firm kick in the backside. I've been there 30 years ago and learned that lesson the hard way. As with everything live within financial means, for some that means a visit once a month, others once a day and twice on Sunday
Myself I find that I often don't do this for weeks or months, too busy with work and hobbies (building model dioramas, aviation) and that's just fine. So to the point, if inflation goes up and financial means become less, adjust accordingly.