Better Tay

If youve ever tried to have a conversation with a chatbot, you know that even todays state-of-the-art systems arent exactly eloquent, regularly doling out nonsensical orpainfully generic responses.

Now, though, Google has created Meena, a chatbot it says is better than any other its tested a claim the company supports using a new metric it developed specifically to measure an AIs conversational abilities.

After creating Meena, a process detailed in a paper published on the preprint server arXiv, Google needed a way to evaluate the chatbot. To that end, it developed something it calls the Sensibleness and Specificity Average (SSA).

To compute this metric, Google asked human workers to conduct about 100 free-form conversations with Meena and several other open-domain chatbots. Each time the chatbot responded, the worker had to answer two questions about the response.

First, did it make logical and contextual sense within the conversation? If yes, they then had to answer the question, Was it specific to the conversation? This was to weed out any generic responses for example, if the human wrote that they liked tennis, and the chatbot responded, Thats nice, the response would be tagged as not specific.

Google determined that an average human would achieve an SSA score of 86 percent.

The other chatbots in the teams study scored between 31 percent and 56 percent. Meena, however, scored a 79 percent putting the AI closer to the level of conversation expected from a human than another chatbot.

READ MORE: Meena is Googles attempt at making true conversational AI [VentureBeat]

More on chatbots: Taylor Swift Reportedly Threatened Microsoft Over Racist Chatbot

Here is the original post:

Google Says Its Chatbot Is Capable of Near-Human Conversation - Futurism