On Sunday (5/27) as the 76th and final match of the IPL was being played, the txtWeb platform experienced an outage.
The duration of the match was about 4 hrs (from 8 pm IST to 12 am IST). At around 8:27 pm we started experiencing issues where many of the user requests started experiencing delays in response or receiving no response, this lasted until the end of the match. We got alerted about this immediately as expected from our end-to-end monitoring devices.
- Approximately half of our user base accessing txtWeb during the match using our primary aggregator (including our primary number 9243342000 as well as several local numbers). Based on expected traffic this would be close to 125k users. (exact figures are awaited from our SMS aggregator)
There were two things that went wrong:
Platform slowness due to database bottlenecks:
Latency on outgoing messaging on our aggregator:
We are overwhelmed with the number of requests we got from our users on that day, seeing peaks of 18K per minute from across India. Our sincere apologies to our users who tried accessing txtWeb during the final IPL match. This was an important learning experience for us and one which will help us improve and continue to strive to delight our users.
|Arjun, Thanks. The platform does support caching for app responses. Please see: http://www.txtweb.com/tutorial...-your-apps Feel free to post in the forum in case you have questions on how it works.|
|Wow, 18k per min is a LOT of traffic. It's pretty impressive. I am curious to know if there was any delay in the response from the apps and if caching of response is done by txtWeb Platform? If there were some delays from the apps caching (on txtWeb platform) can help reduce the bottleneck. Cache can be invalidated after 30-45 sec (after each delivery is bowled). In this way the response for 9000-13500 users will be same and will improve performance. If caching is already supported then It'd be great if APIs for developers are opened up, so that we can set the time interval for cache invalidation.|
Additional Info: Other
In this issue, we bring you a basket of apps that can make those rainy days really special. W ..more >
Shakespeare had said, “If music be the food of love, play on..” and we couldn’t agr ..more >
Tis the season to be jolly! Well December is already here and Christmas and the New Year is just a fe ..more >
Lately, I have been using @omegle app a lot. To send a message, I had to precede my text with "@omegle.msg" e ..more >
Shot to fame in App2Fame owing to his app @smsgps (which detects current location of a user ..more >
When we spoke about our last platform and community updates in June this year, we promised you that t ..more >