How does Tinder develop the features that keep you safe?

Who decides what we need to stay safe online? And how do they know what features we would benefit from?

One person who has played a key role in the dating app’s security features at Tinder is Rory Kozoll, Tinder’s senior vice president of product integrity. Kozoll leads the team that develops in-app tools and resources aimed at making user interactions respectful and safe.

Tinder has introduced a number of new security updates and features, most notably a long-press reporting feature that lets you tap and hold chat messages to directly initiate the reporting process. This means it’s now easier to report harassment, hate speech, or other offensive text that violates the app’s Community Guidelines(Opens in a new tab).

According to a recent poll conducted by Opinium on behalf of Tinder, 72 percent of 18-25 year olds are as concerned about their emotional safety as they are about their physical safety. The survey, which looks extensively at online interactions, also found that 40 percent of 18-25 year olds have seen hate speech online and 30 percent of people admit to sending harmful messages online that they later regret. Additionally, Tinder is expanding its existing ‘Do you mind?’ and ‘Are you sure?’ Features to expand the categorization of hate speech, harassment, and sexual exploitation.

For women and the marginalized gender, being on dating apps, social media, or just being online in general can be associated with sexual harassment, receiving non-consensual, unwanted sexual messages, in addition to leading to violations such as cyberflashing.

How does Tinder know what security features users need?

Kozoll spoke to Mashable about how Tinder’s security tools are developed and what four main sources of information go into the process.

“Our members will tell us that something has been troubling them, and that will give us the signal that we need to step up and try to understand what the offense might be and how we can help reduce that offense,” he says he. “The second source is the things that we can see very clearly in our data. And the third is that we work with many external partners, both in the gender safety space and in the LGBTQIA space and other underrepresented groups, to inform us.”

The fourth source is “a bit more art than science,” says Kozoll, meaning “product intuition”. Tinder’s own staff use the app and will report and discuss their own experiences to share what they think needs to change on the platform.

Tinder’s “Do you mind?” feature comes from a real-life experience.

In the case of Tinder’s “Does This bother you?” feature, a real-life incident led to this tool being introduced to the app. The tool uses machine learning to flag potentially offensive messages and triggers an automated message to be displayed to message recipients when malicious language enters a conversation. With this prompt, users have an immediate opportunity to report the bad behavior if they choose to do so.

Want more sex and dating stories? in your inbox? Sign up for Mashable’s Top Stories and Deals newsletters Today.

Before releasing this feature, Kozoll and his team had researched categories of offensive messages. When it comes to what Kozoll describes as “more forward talk” (read: sexually explicit messages), the key factor to consider is consent.

“People can open the door, say, more forward talk. We want to make sure we always walk the line between everyone’s safety and everyone’s well-being and not imposing ourselves and our own values ​​on our members,” he says.

Kozoll says he and his team are constantly monitoring real-world examples of issues people may encounter in the app.

“I was having dinner with my wife and I was walking to a restaurant in Santa Monica. This car drives by with these young guys and one of them leans out the window and shouts. As I turned I could see that there was a young woman of her own walking behind us. You could just see that she was visibly uncomfortable when the guys were whistling,” he explains. “They kept driving and out of instinct I just turned around and said, ‘Hey, are you alright? Do you want to go with us?’ Turns out they went to the same restaurant.” At that moment, Kozoll’s wife said to him, “You don’t know how seldom it happens that someone really just asks, ‘Are you okay?’

“That was the seed — just because we don’t know for sure if those messages are problematic for that person, it never hurts to just ask them if they’re okay.” comes from,” he adds.

What role does Tinder want to play here?

When it comes to the challenges the Tinder team faces in addressing security requirements, Kozoll says, it’s all about “figuring out where the right line is between making sure everyone feels comfortable, but also giving them freedom to express themselves and have the kind of conversation they want to have.”

“We see ourselves as hosting a party and we invited all these guests. We hope people get along well and meet someone exciting and new. We’re not there to tell people how to talk, but we’re there if someone looks across the room and gives us a look to say, “Hey, I’m really uncomfortable here.” We have to intervene and help resolve the situation. Sometimes that means asking someone to leave the party, and that’s the role we’re trying to play,” he says.

So, why has Tinder upped the ante when it comes to hate speech? Kozoll says it has to do with how language develops in society.

“Language is always evolving, emoji is always evolving, people are getting more creative, they’re not trying to evade what we’re doing. But only the language is constantly changing, and that’s why we have to adapt to it very quickly,” he says.

“As we continue to develop our understanding, we will continue to update these models,” adds Kozoll. “This is a never-ending stream of work, evolving these machine learning models and keyword lists so they can better identify the context in which those words appear and the new words that also appear in the lexicon.”

Read more about staying safe in the online dating world: