Text-based question and answer services are the latest way to look clever. For about £1 you can text a question and a few minutes later receive an answer. But we've heard that some of the answers sent by some of the biggest question and answer services are wrong.
Watchdog looked at three big names in the market: 118 118, which is the market leader; Texperts, which is now owned by 118 118; and AQA 63336. The AQA stands for 'Any Question Answered'.
Ben Archer is a keen rugby fan, and relies on texting 118 118 to find out local rugby results because he doesn't have access to the internet over the weekend. But when he texts them asking about his team in Bury St Edmunds, Suffolk, they text him answers about a team in Bury, Lancashire.
He says he realises no one's perfect, but that if 118 118 is providing a service it should be accurate.
Watchdog experiment
To test the companies' accuracy, Watchdog conducted an experiment last week. We texted 20 general knowledge questions to 118 118, Texperts and AQA 63336. We sent the same questions twice in case the companies replied with an incorrect answer the first time. So 40 questions were asked in total.
The results showed that Texperts sent us the correct answer to the 40 questions only 65 per cent of the time. On some occasions it told us it didn't have enough information to answer our query. Once, the answer it gave seemed to be for a completely different question we hadn't asked.
118 118 correctly answered the 40 questions 72.5 per cent of the time. It sometimes answered one question with a wrong answer, but when we texted the company again it sent us the right answer. On one occasion it gave us an answer that we'd made clear in our question we didn't want.
AQA 63336 were the most accurate of the three companies, answering 92.5 per cent of the 40 questions correctly.
When 118 118 and Texperts can't give you an answer they won't charge you. AQA63336 will give you a refund. If you can prove the answer they've sent you is wrong, all three will give you your money back.
Celebrities versus the companies
We tested three clever celebrities against the answers we'd been sent by the companies. We assembled a crack team comprising quiz maestro Keith 'Cheggers' Chegwin, Cambridge graduate Konnie Huq and all-round know-it-all Iain Lee. We asked them the same questions, though we only asked them once rather than twice. They also had access to the internet, as the text companies do.
Our celebrities didn't beat AQA 63336, but they did answer 18 out of 20 questions correctly, giving them a score of 90 per cent. Better than both Texperts and 118 118.
AQA 63336's response
We've 1,000 mostly UK based-researchers who together answer an average of 15,000 questions a day, and have seen peaks of up to 25,000 questions a day. Since launching AQA 63336, the world's first text Q&A service on a premium text code, we've answered over 17 million questions from 2 million customers.
As a company we always try to provide great customer service, and as a company our top goal has always been to provide a high quality service. All questions texted to AQA 63336 are answered by human researchers, who strive to provide not only the right answers, but answers written in the Queen's English in a way that's warm, positive, and often witty. By being a human-based service we provide great value to our customers, but we've always recognised that because it's a human-based service, we'll make mistakes. There are, of course, some questions where there is no right answer, or answers are disputed. However, if we do get the answer wrong, we will always apologise, refund the customer, and importantly correct the information that's been given out. As a note, we currently only get one customer complaint per 10,000 answers sent out.
As we've said, all questions texted to AQA 63336 are answered by human researchers, who strive to provide brilliant answers - answers that are fast, pithy and accurate, but always fill up a text message, and often provide additional information that will wow our customers. There are occasions where it's possible to have more than one right answer to a question, so sometimes we do give different answers. As mentioned earlier we are a human service so also we can occasionally get it wrong.
Response from 118 118 and Texperts
Due to the recruitment of new staff, less experienced employees have made some mistakes. This is being addressed by further training and supervision.
Our own internal quality control shows that we're achieving more than 90 per cent accuracy. The vast majority of questions carry no ambiguity. The clever questions posed by the BBC simply do not represent the actual range of straightforward questions received by 118 118 from the public.
We acknowledge all questions but do not reply to certain questions that incite defamation, racism or other forms of discrimination, or for information that might constitute civil liberty issues, unwarranted invasion of privacy, or for information such as medical or financial advice that requires specialised, qualified advice.
In the event we're quite unable to answer a question, we do not charge for our response. Note that this is different to AQA, where the user is automatically charged for asking the question, regardless of response. We do not charge on the inbound question.
We're very careful to only advertise Ask US Anything, and not Any Question Answered, because we recognise not all questions can be answered. We believe we are being wholly honest in our claims and up front about what we can and cannot do. We further believe the sweeping advertised claim of AQA, Any Question Answered, would require a score of 100 per cent to meet expectations.
We're proud of our new service, which has become the market leader very quickly. We accept there have been some teething troubles, but believe the service is delivering an important service to the British public, and look forward to ever higher levels of accuracy and usefulness.
Text question and answer companies
Watchdog looked at three big names in the market: 118 118, which is the market leader; Texperts, which is now owned by 118 118; and AQA 63336. The AQA stands for 'Any Question Answered'.
Ben Archer is a keen rugby fan, and relies on texting 118 118 to find out local rugby results because he doesn't have access to the internet over the weekend. But when he texts them asking about his team in Bury St Edmunds, Suffolk, they text him answers about a team in Bury, Lancashire.
He says he realises no one's perfect, but that if 118 118 is providing a service it should be accurate.
Watchdog experiment
To test the companies' accuracy, Watchdog conducted an experiment last week. We texted 20 general knowledge questions to 118 118, Texperts and AQA 63336. We sent the same questions twice in case the companies replied with an incorrect answer the first time. So 40 questions were asked in total.
The results showed that Texperts sent us the correct answer to the 40 questions only 65 per cent of the time. On some occasions it told us it didn't have enough information to answer our query. Once, the answer it gave seemed to be for a completely different question we hadn't asked.
118 118 correctly answered the 40 questions 72.5 per cent of the time. It sometimes answered one question with a wrong answer, but when we texted the company again it sent us the right answer. On one occasion it gave us an answer that we'd made clear in our question we didn't want.
AQA 63336 were the most accurate of the three companies, answering 92.5 per cent of the 40 questions correctly.
When 118 118 and Texperts can't give you an answer they won't charge you. AQA63336 will give you a refund. If you can prove the answer they've sent you is wrong, all three will give you your money back.
Celebrities versus the companies
We tested three clever celebrities against the answers we'd been sent by the companies. We assembled a crack team comprising quiz maestro Keith 'Cheggers' Chegwin, Cambridge graduate Konnie Huq and all-round know-it-all Iain Lee. We asked them the same questions, though we only asked them once rather than twice. They also had access to the internet, as the text companies do.
Our celebrities didn't beat AQA 63336, but they did answer 18 out of 20 questions correctly, giving them a score of 90 per cent. Better than both Texperts and 118 118.
AQA 63336's response
We've 1,000 mostly UK based-researchers who together answer an average of 15,000 questions a day, and have seen peaks of up to 25,000 questions a day. Since launching AQA 63336, the world's first text Q&A service on a premium text code, we've answered over 17 million questions from 2 million customers.
As a company we always try to provide great customer service, and as a company our top goal has always been to provide a high quality service. All questions texted to AQA 63336 are answered by human researchers, who strive to provide not only the right answers, but answers written in the Queen's English in a way that's warm, positive, and often witty. By being a human-based service we provide great value to our customers, but we've always recognised that because it's a human-based service, we'll make mistakes. There are, of course, some questions where there is no right answer, or answers are disputed. However, if we do get the answer wrong, we will always apologise, refund the customer, and importantly correct the information that's been given out. As a note, we currently only get one customer complaint per 10,000 answers sent out.
As we've said, all questions texted to AQA 63336 are answered by human researchers, who strive to provide brilliant answers - answers that are fast, pithy and accurate, but always fill up a text message, and often provide additional information that will wow our customers. There are occasions where it's possible to have more than one right answer to a question, so sometimes we do give different answers. As mentioned earlier we are a human service so also we can occasionally get it wrong.
Response from 118 118 and Texperts
Due to the recruitment of new staff, less experienced employees have made some mistakes. This is being addressed by further training and supervision.
Our own internal quality control shows that we're achieving more than 90 per cent accuracy. The vast majority of questions carry no ambiguity. The clever questions posed by the BBC simply do not represent the actual range of straightforward questions received by 118 118 from the public.
We acknowledge all questions but do not reply to certain questions that incite defamation, racism or other forms of discrimination, or for information that might constitute civil liberty issues, unwarranted invasion of privacy, or for information such as medical or financial advice that requires specialised, qualified advice.
In the event we're quite unable to answer a question, we do not charge for our response. Note that this is different to AQA, where the user is automatically charged for asking the question, regardless of response. We do not charge on the inbound question.
We're very careful to only advertise Ask US Anything, and not Any Question Answered, because we recognise not all questions can be answered. We believe we are being wholly honest in our claims and up front about what we can and cannot do. We further believe the sweeping advertised claim of AQA, Any Question Answered, would require a score of 100 per cent to meet expectations.
We're proud of our new service, which has become the market leader very quickly. We accept there have been some teething troubles, but believe the service is delivering an important service to the British public, and look forward to ever higher levels of accuracy and usefulness.
Text question and answer companies