It's not popular because throughout the history of the game the administrators have done everything they can to exclude others from playing.
The earliest days it was just a sport for the rich elite and excluded the working classes, then after colonization it was just a sport for the white man. Indigenous populations and women were not invited to pay, and often were literally banned from the grounds.
It took ages for cricket to become inclusive of all races and genders (tho tbf one can argue they were ahead of the curve when compared to several other sports), but by that point the major cricket boards were now more interested in excluding smaller nations from playing the game at the highest level, let alone involving them in the running and administration of it.
That was changing for a brief while, and there was more emphasis put on Associate cricket tournaments and developing cricket in new regions. Nations like SL, BD became full members, the ACC was set up with Associates as the founding members, and we had tournaments like the WCL and Intercontinental Cup. But then the Big 3 takeover happened. Forget new nations, the existing non-Big 3 ones were now struggling. In 2007 we had 16 teams in a World Cup, but by 2019 we only had 10. We're the only sport that is shrinking our World Cup.
Cricket already is a difficult sport at the best of times to introduce to people unfamiliar with it. The laws are complex, you need a lot of resources to play it (compared to other sports), and we have three different formats of the same sport essentially competing for the attention of a single market. It does not help when the Administrators in charge are more interested in making a quick buck through broadcast deals and petty political bickering rather than actually trying to grow how many people engage with the sport.
Despite that all metrics show cricket to be amongst the Top 5 sports in the world. Yes there are a billion Indians that account for this, but even then it's extremely popular in a lot of nations, and it's growth in the past few decades has been solid. Stories like Afghanistan, Thailand women and Sandeep Lamichhanne are evidence of this.
Cricket is just fundamentally a great sport, we just need administrators who believe in it and are willing to get out of it's way. However this is unlikely given how the global game is currently administered.