The Challenge: Is 100 % coverage possible in chatbots, furthermore in mobile Chatbots?
As per testing nomenclature 100% coverage is never possible in chatbots. As a chatbot basically is trained to give specific responses as per the user’s inputs which in technical terminology is quoted as “Utterances”.
So, it is next to impossible to train a chatbot for all available variations of a specific utterance. For example, in English language alone there are more than 16 ways to just ask a person “How are you”, so 100% coverage would mean training the chatbot for each one of those, sounds cumbersome, doesn’t it? it isn’t exactly possible to predict the kind of questions a user may ask or the way in which it may be framed by the user. Small talks by user, irrelevant statement requests, and some out of domain questions are some of the vital things to be noted as “must-have” features of bot.
When these trainings are implemented in Mobile bots, there’s an estimation that comes into picture. As of 2018 there are approx. 3 billion smartphone users in the world: that’s about 40% of the world’s population out of which there are almost 400 million users in India alone.
So that’s the biggest target audience that one can cater to. Apart from users there are more than 30,000 different android devices alone each of them having different display sizes, technical specs etc., and if we talk about IOS the variations in specifications and display sizes are even more as apple provides display sizes from 3.5 inches iPhones to about 12 inches iPads. It’s a big challenge of maintaining uniformity in the look and feel of the application or the chatbot in all these devices throughout.
In mobile testing always we face device specific issues, i.e. an issue arising in one device might not exist in other. Furthermore, apart from devices and screen sizes; there are 2 more factors in mobile devices which can be a challenge for testing: Manufacturers and Operating Systems.
As of 2018 there are more than 2000 companies which manufacture smartphones, using different hardware and software platforms as per convenience. Devices also have many operating systems on which they run. For example, in android devices each manufacturer has its own OS which is a customized version of the stock android OS. Another challenge with apple devices is that few old apple devices such as the iphone4 have stopped receiving support from apple because of its operating system version.
Taking all this into account we can imagine the complexity of the process of chatbot testing in mobile and the hardship that the tester will face. Our goal is to overcome such challenges and gear up the walking bot. The Ride is surely going to be tough, but it shall be worth it.