AI Technologies , Generative AI , Large Language Models

Artificial Intelligence Is Not a Product, It's a Journey

AI Is a Journey That Requires Continuous Learning Muqbil Ahmar • February 1, 2019

The Bombay Stock Exchange (BSE) has been at the forefront of adopting new technologies. From being the first innovators to use the surveillance-as-a-service model to migrating to open source and using artificial intelligence (AI) in novel ways, the organization has set new benchmarks during its digital journey. It has leveraged the digital to enable business by getting into new markets and providing greater security for transactions. Kersi Tavadia, CIO, BSE, shares the organization's plans as well as the challenges that lie ahead.

Edited excerpts follow:

Trust and transparency are critical to BSE's operations. How is technology helping the exchange adhere to a rigorous compliance framework?

Our market surveillance platform has come a long way in bringing high levels of integrity and transparency in the market. It is an instrument of trust. We were the first innovators to use the platform for surveillance-as-a-service model. We developed our expertise to implement the exchange and then extended it to our broking members at zero cost. While the exchange monitors the brokers' behavior, the brokers, in turn, monitor their clients'. We send them regular alerts and flag any break in pattern in real time, which raises an alarm. It is an auto-ticketing system, and the entire action is workflow based. The broker has to close the ticket either as a "false positive" or "action taken" so that it is tracked. The alerts are customizable and set by brokers. This has ensured transparency and trust between market participants and investors.

Data is the new oil. How have BSE's technology systems evolved over the years to leverage data better?

BSE has been a collector of proprietary and top-of-the-line technologies. However, even these advanced technologies were difficult and expensive to maintain and upgrade. As trade volumes and transactions started increasing, expanding the traditional data warehouse became prohibitively expensive. Also, a lot of systems were operating in silos. We had to break the data silos and bring it to one place to provide a single version of truth. If the data is stored in multiple systems, there are lots of discrepancies. The task was to collate the data into one system. We carried out a data warehouse migration activity and made the shift from proprietary technologies to Apache Hadoop.

Secondly, we were able to converge all our systems. For all our reporting systems, we tried to convert them into a web. We created a data lake so that all our regulatory reports happen from the data warehouse system. All transactional reporting happens from individual systems. So, we have the data at a place from which historical reports can be made. My transactional systems have become lighter and other costs have also gone down. When you have such large data, you have to back them up and retain them. There are so many costs involved. Even when you want to restore, it is time consuming. You may require a small amount of data but the data warehouse is so large, it might take days to acquire it. So, there are a lot of costs involved when you don't streamline or consolidate your data. It was a migration strategy and cleansing of data. Then we started building analytics to use the data.

What's your topmost priority to enable technology adoption?

Migration to the cloud wherever possible with focus on two areas - cost optimization and security. Cloud has its own advantages and disadvantages. BSE's application models do not warrant too much on cloud presence. We retain a majority with us. But wherever possible, we try to leverage the cloud. We are doing a lot on the AI front. We are also planning to build a cybersecurity operations center for financial markets.

What key initiatives are you undertaking around AI?

BSE is using AI for real-time surveillance. The technology also has a 95%-97% success rate for fraud detection and rumor analytics. This is how AI is helping BSE put checks and balances in place. If there is any news about a listed company, the surveillance department generates an automatic email to the companies informing them of the development. Whatever clarification the company gives is immediately put out on BSE's website.

While earlier, this used to happen once a day, with all the automation, it can be done every 15 minutes. Besides this, we are also doing a lot of high-powered computing from social media and working on speech recognition on the AI front. For instance, the exchange records media interviews. If somebody is giving a live interview, the system can immediately correlate it with the market, inform how it has impacted the market, and show the trending analysis. AI is never a product. It is always a journey with continuous learning. If you are not learning continuously, then you can't build intelligence.

This article was previously published on the DynamicCIO website on Feb. 1, 2019.