AI Readiness Index – Where Are You on the AI Journey?
A couple of months ago, AI Singapore unveiled the AI Readiness Index (AIRI), a comprehensive and easy-to-implement industry-focused framework which enables organisations to self-assess the status of their artificial intelligence (AI) adoption readiness. I talked to Tern Poh from the AI Advisory Team, one of the architects of the framework.
Below is a transcript of the conversation [*].
Hi Tern Poh. How are you today?
Hi Basil, thanks. I’m doing well today. I’m staying at home, working from home. Miss the outdoors. Otherwise, I’m doing fine.
Yes, we all have to stay safe during this period. So, today we will be talking about how to improve the journey of AI adoption in industry. According to a survey done by Gartner, the top three challenges to AI adoption are : (1) skills of staff, (2) understanding AI benefits and uses and (3) data scope or quality. As part of your job to get companies to adopt AI, you go beyond identifying these factors and empower companies to quantify them in order to better understand their AI adoption readiness level. That’s where AI Singapore’s AI Readiness Index (AIRI) comes in. As one of the architects, tell us more about what AIRI is.
So, AIRI in brief is an industry-focused AI readiness assessment framework. So it allows the business units or organisations to assess their AI readiness, meaning where they are right now, and with this, by understanding where they are, it will enable them to identify the gap between their current and desired state. With the gaps identified, it will enable organisations to have a better understanding of where they need to work on in order to improve their AI readiness and to close up the gap. This is important because a lot of times organisations want to embark on a new business initiative and that business initiative might require certain AI capabilities in their organisations for them to achieve that. So without having an assessment of where they are, the organisations might not be able to reach their business objective at the end of the next financial year, for example. So that’s why AIRI is designed for. It helps the company to understand their current state of readiness and to close up the gap between the current and the desired state.
I think it’s a very relevant initiative, and just to go back into the history, what inspired and drove the development of AIRI?
I think AI Singapore, we are in a very fortunate position because we are a national programme office. So, we get to work with companies of different sizes, from start-ups to MNCs and across different industries as well, so we are agnostic. Whichever company or industry needs our help, AI Singapore will be there for them to help them accelerate their AI adoption. Since AI Singapore started a few years ago, we have engaged more than five hundred companies across different sizes and industries, to have discussions with them to understand how we can help them to accelerate their AI adoption. It is through these multiple discussions with companies we started to realise that there’s a recurring theme among these companies. And these companies often have questions for us.
The first question to ask is, how do I know whether I’m AI-ready? This is the question that a lot of companies have at the top of their minds. And sometimes it is also pretty funny, like when we talk to the clients, based on our assessment, hey you’re not data-ready yet. You have to come back to us once you’re more AI-ready, for example. Then in the next question they ask is, how do I know when I’ll become AI-ready? How do I assess it? How do I quantify all these things that you mentioned earlier on? So there’s a recurring theme between these questions from the companies we’ve engaged and that really drove the inspiration and the development of AIRI. We want to create a systematic method or framework to really help companies to understand and to quantify their AI-readiness so that we are able to help them improve it.
And as an organisation that is data-driven, that emphasizes being data-driven, I think this is a natural development. After all, we’ve been engaging with already more than five hundred companies and organisations. But, AIRI, it’s not the first AI-readiness on maturity index. What makes it special compared to the others?
Ya, that’s a very good question. If your google AI readiness index or AI maturity index, you get a lot of returns from the google search. Back to the earlier question when we had a lot companies coming to us, asking the same question, we went out to the Internet and looked at some of the existing indexes or frameworks developed by others. I mean, why re-invent the wheel when we can just adopt it. But, surprisingly, we soon realised that none of the existing AI readiness or maturity index has what we need. I won’t mention names, but some of the existing frameworks that we saw are too complicated. Too complicated in the sense that it goes into areas that we don’t think are needed to determine whether a company is ready for AI adoptions. Things like the HR system, the procurement system, etc. So, some frameworks go way beyond what is really needed for AI adoption. So, these are some of the frameworks that we see. They are too complicated and if they are too complicated, it’ll be hard for any company to do a self-assessment.
This brings me to the next point. There are some other frameworks that we see that are proprietary, meaning some companies have developed a methodology, a framework around how they can assess somebody’s AI-readiness, but you have to pay them a hefty fee to engage their consultants to do the assessment on behalf of the companies. When we are trying to develop AIRI, our intention is to help, especially the local SMEs, the small-, medium- enterprises, to help them accelerate their AI adoption. And I think it’s very hard for SMEs to pay such a high amount of consulting fees to assess their AI readiness. So, given all these backdrops, when we came up with AIRI, we really wanted something that is very easy to implement in a way that companies can do by themselves without any consultants and the framework is simple enough, yet comprehensive to look at all the factors that are really core to AI-readiness, instead of looking at every single aspect of an organisation. So, this is what we see in industry and what drove us to develop AIRI to really help companies accelerate their AI adoption.
So, what we can expect from AIRI are : cost-effectiveness, ease-of-use and something that’s really actionable, right?
Ya, and more than that as well. Other than looking at what is out there, a lot of it is really based on our core learnings. Again, AI Singapore we have engaged a lot of companies across different industries, like I mentioned earlier. So, we have this knowledge of, or you can say a gut feel of how we determine whether a company is AI-ready or not, but we have not formalised it into a framework. So, AIRI is really a formalisation of all our learnings of our past engagements with the companies. It goes beyond referencing what is available. It really incorporates and distills the core learnings and experiences that AI Singapore has gathered over the past few years.
Now I’m eager to get into the details. How does it work AIRI work?
I’ll go through from the high level first, to the finer details. AIRI essentially consists of four main pillars which map to nine dimensions. By dimensions I mean the areas of assessment that we are looking at to quantify whether a company is AI-ready or not. So, the first pillar, perhaps the most important pillar, is the organisational readiness. It looks at four main dimensions. We’re looking at AI talent, AI literacy, AI governance and management support. For AI talent and literacy, they are pretty self-explanatory and straightforward, so I won’t elaborate further on them. What I would like to elaborate on is management support. A lot of times when people think about AI readiness, they think about getting the latest equipment, about hiring a data scientist or sending their employees for courses to improve their AI literacy. All these are things that people typically look at, but what they miss out on is management support and that’s a big mistake. What we have realised is that over the years, for companies without strong management support, it is hard to get projects started and even if the project managed to get started, halfway through it will get derailed. Let me share with you more insights about this. AI is a very powerful tool and when you want to embark on an AI project, typically it involves collaborations across different departments or divisions of business. Without a strong mandate and support from the management, the different divisions or departments wouldn’t be that keen to come together to form a cross-functional team to do a project together. This is what we see typically. AI when it is deployed in the organisation as well, it requires a change in the way that the employees and the organisation functions. So maybe in the past, you use Excel sheet to do some predictions. But with AI the workflow will change. So if you change the workflow, the impact will be felt across different departments. So again, without strong management support, the different divisions or departments in the company might try to sabotage the project because they do not want to change their existing processes. I mean, why change their current processes just to adopt a new technology when it disrupts their workflow, for example. So, that’s why I think management support is very critical.
And the one last dimension about organisational readiness is AI governance. So, we added that in because right now there’s a trend in the industry where people are no longer talking about how accurate is your AI model. I think the industry has moved beyond that. Yes, of course, accuracy of an AI model is important, but what is also important right now is whether the AI model developed is trustworthy, whether it is ethical and whether it is responsible in the way that is being applied. Because AI is a very powerful tool and if it is applied wrongly, it might have unintended consequences for the organisation or for society at large. So, for an organisation to be AI-ready, AI Singapore looks beyond what the technical capabilities are. We also look at the management support and whether they are going to apply AI in the right manner. So, sometimes when we see companies that come to us and they want to apply AI in a manner that we think might not be suitable or might be sensitive, we will tend not to do the projects with them and we try to educate them what are the consequences of doing such projects, so hopefully they also understand about this point.
The first pillar is organisational readiness, so what about the second pillar?
Okay, the first pillar was a bit long, because I think it’s important to share the rationale of why we put certain things inside. I think for the second pillar, it’s more straightforward. For the second pillar, it is a business value readiness. It has only one dimension, which is business use. Overall, this pillar looks at whether the organisation has identified business use case for AI adoption and the potential value it brings to the organisation. So, AI it is just a tool, a technology. Just as any other tool and technology, it can only bring value to the organisation where it is applied in a manner that is relevant to the organisations. A lot of organisations tend to apply AI for the sake of applying AI. That’s not the way we recommend companies to proceed forward to adopt AI. We always ought to anchor AI adoption around business use cases to justify the time and investment required to begin an AI project. I think that’s number one, and number two is that with a strong business use case, it serves as a rallying point for employees across different divisions or departments in the organisation to come together to work on a project together. That’s why the business use case is so crucial. It serves as a justification for the management to invest the time and effort in doing the project and it helps to gather employees across the organisation to come together. This will help to address some of the issues I mentioned earlier.
Just to sidetrack a little, how do you determine the business value of a use case?
That’s interesting also. When we talk to customers or companies, most of the time they will ask us, so what will be the expected value that we will get if we are to work on this project with AI Singapore? For example, it could be things like predictive maintenance or customer segmentation. As much as I would like to give them an answer, the reality is it’s very hard for an AI engineer or technologist to advise the business people what will be the expected ROI or business value of a use case, because only those who are deeply involved in their organisations or who is very familiar with the industry would know what value it will bring when this particular issue is solved. And that’s the reason why for most of the programmes under AI Singapore, we typically require the organisation to nominate a key person that we can speak with throughout the project because this domain expert would know what is the business use case and how should we go about doing the projects. Because if you are just going to apply AI to the datasets, there could be a lot of unintended consequences in the way that the model is being built to the way that are we assess the business value. So, we always require the businesses stakeholder to understand and assess what is the value of each use case. We are not able to advise that.
Okay, so coming back, we covered the first two pillars: organisational readiness and business value readiness. How about the last two?
So, the third pillar is on data readiness. We are looking at the reliability, quality and consistency of data throughout the organisation. Under this pillar, we have data quality and reference data. I won’t go into the details of each, but one question I often get asked is, hey Tern Poh, when you are looking at data, I thought it should be with respect to a specific use case. So, how can AIRI assess the data readiness of my organisation without me telling you what is my specific use case? This is a common question I get asked whenever I share with companies on AIRI. My short answer is this. AIRI looks at an organisational level, meaning at an organisational level, what’s the likelihood that this organisation is ready to adopt AI. We do not go down to each specific use case yet. Of course, we have other ways to look at to assess the data with respect to each use case, but this is not the intention of AIRI right here. For AIRI, we are looking at, at an organisational level, what is the tendency that this organisation is data ready. So, in the questions that we ask for the data quality and the reference data, the two dimensions under this pillar, we are looking at whether the company has policies, processes and employees who are actively maintaining the data quality. The thing is this. For organisations who have formal processes and policies in place to ensure data quality, then it is more likely that for any use cases this company is intending to pursue, there is a higher likelihood that these use cases will have the data ready. So, this is how to we look at it. At an organisational level, whether there are formal processes and policies in place to ensure data quality and reference data.
And the last pillar of AIRI is infrastructure readiness. It looks at whether the organisation has the necessary infrastructure to support data storage, retrieval and AI model training. Briefly put, we are looking at whether the company is using appropriate methods of storing data, so if the company is still using paper to record transactions, not in an electronic format, that is very hard for us to do anything for the company. So, that’s the data infrastructure part. For the machine learning infrastructure part, we’re looking at whether the company has the right equipment to train the AI model and to deploy the AI model once it is ‘live’. So, these are the four main pillars of AIRI.
That is really a very comprehensive assessment being done. At the end of the whole process, what are the ways you see companies can benefit from using AIRI?
I think this is a very important question. After going through this AIRI assessment, how do I benefit from it? At the end of the assessment, the company will receive a report, a customised report based on their AIRI result. The result will classify their company into four different readiness state.
From the lowest, that will be AI unaware, to the highest, that will be AI competent. So, we basically classify companies into four different categories. Under each category, we will have a recommendation to the company in terms of what AI adoption approach it could take.
In the lower two AI classifications for AIRI, we have the AI unaware and AI aware. These are the companies that we think are able to adopt AI. In fact, any company could adopt AI. It doesn’t mean that, no, if I am AI unaware, I am not able to adopt AI technology. For such companies, they could also adopt AI technology by buying something that is commercially available off-the-shelf just by installing the software on their system, they can start using the AI application. Every organisation could benefit from adopting AI, that is one thing I want to highlight. For the lower two categories, AI unaware and AI aware, for such companies they could adopt AI. Just that they shouldn’t be spending their efforts to develop their own AI model. They can looked out to the industry to see what’s out there and to buy such solutions.
In the higher two categories, we have the AI ready and the AI competent. What separates these two is that for AI ready companies, typically for such companies they have the software engineering capabilities and for such companies they could make an API call to a pre-trained AI model or to one of the cloud providers’ API services to integrate AI features into the product. So, this is what we see for companies which fall under AI already. For AI competent – the level four – for such companies in the AI competent region, they are able to develop their own customised AI if none exists in the industry. These are the companies which develop AI solutions to meet their internal needs, it goes beyond just looking at what is available in the industry.
One thing I want to highlight is that AIRI does not insist that every organisation needs to be at AI competent. People have this misconception that if I’m AI unaware, my end goal should be AI competent. That’s not what we are trying to emphasize. What we are trying to emphasize here is that AIRI helps you to understand where you are right now, at your current state and where you should be should be based on your business objective. Let’s say, for example, if a business or an organisation, based on their business objective, they are not intending to build their own AI engineering team. All they want to do is to start integrating or buying some AI solution to solve their internal needs, maybe all they need to do is to be at AI aware stage. It becomes very clear, from AI unaware, they can move to AI aware and that is where they need to be. There is no need to go beyond that level if their business objective doesn’t require that. I want to highlight this because not every company is looking to develop their own customised solutions. Sometimes, all they need is just to have some AI features in their products. But, although we say that we don’t recommend every company to be AI ready or AI competent, but what we would like to encourage is that companies should at least be at AI aware stage, because if a company is at AI unaware, they are missing out on a lot of the good things that AI can bring. For instance, for companies who are at the AI unaware, they wouldn’t know what are the possibilities that they can do with AI and what are the typical use cases that they can implement with AI. These are the companies at the AI unaware stage, but once they get to the AI aware stage, at least the organisation has the capability to understand what are the common use cases of AI and whether they want to apply AI or not, that is a separate issue. But at least in the organisation, the management knows what is AI and whether AI can be used in their internal use cases. That is very important because right now, the world is moving towards the age of AI. If a company is happy enough to be at the AI unaware stage, they might be missing out on a lot of the opportunities. So, as much as possible I’ll try to push companies at least to the AI aware stage.
So, by doing this AIRI assessment, companies can get to know where they are in one of four readiness states and with that comes a list of recommendations to guide them in their future journey on AI adoption.
Yes, Basil, you are right. So, even at AI Singapore, if the listener goes to the AIRI website, they will see that we actually map the AI Singapore programmes, such as 100 Experiments, to different readiness level based on AIRI. For instance, for a company who is AI unaware, they will not be suitable for a 100 Experiments, because for 100 Experiments, we have certain prerequisites a company has to fulfill before we will consider them. At AI Singapore, we are also mapping our programmes to each category of AI readiness based on AIRI. So, with the AIRI assessment done, it becomes very clear to the companies what are the programmes that they could take if they want to improve their AI readiness. This is crucial because otherwise, people always talk about improving their company’s AI readiness, but without understanding where they are right now, how do they even identify the right programmes or the right approach to increase or improve their AI readiness? This is where I see the value of AIRI is. It helps companies know where they are right now today and based on their business objectives, which give them their intended goals in the future, with these two points mapped out, then they can identify whether there’s a gap. And if there is a gap, they can go out to look at what are the appropriate programmes that can help them close up each of the gaps identified under the nine dimensions of AIRI.
So, how can companies take the AIRI assessment?
That’s easy. They can just go to the AI Singapore website, there’s a site dedicated to AIRI. They can do it there and it is free. When I say it’s free, it is really free. There’s no advertisement as well, it’s not ad supported. And AIRI just consists of nine questions, nine multiple choice questions. It can be completed within ten to fifteen minutes. Very short and sweet assessment. And at the end of the assessment, the companies will be able to download a customised report immediately, there is no need to wait. Once they are done with the AIRI, they can download the report and look at what AI Singapore actually recommends to them in terms of AI adoptions and AI readiness.
Great. So, before we close, what advice do you have for companies that are looking to adopt AI?
I think for any company who wants to adopt AI, it’s very important to determine the current state of readiness. Where they are today and looked at their business objective, where they want to be in the future. It’s only when you have these two goals in mind, then the company can chart out a journey of how they can get from where they are right now to their destinations. So, once you have a journey mapped out, it become clearer to the organisations what are the things that the organisations need to take. For example, if you go on a journey to the supermarket near your neighbourhood versus a journey to another country, it could be a road trip, the journey distance, the journey itinerary, the journey equipment that you need to bring along will be totally different. So, if you put that analogy back to the question, by knowing the current state of readiness, the future state or the destination the company needs to be at, once you have these two mapped out, it’s then crystal clear to the companies how long will the journey take, what are the talents, the training programmes that these companies need to bring in in order to help them achieve their goals and succeed in their journey. Otherwise, it will be very hard, if you ask anybody to plan for a journey without knowing the destination. It’ll be impossible to plan for the journey. And, of course, please do the AIRI assessment today.
Yes, of course. Our listeners out there, if you are leading an organisation or wanting to get onto the AI journey, just go to https://aisingapore.org/airi/ to get your assessment done today. Thank you, Tern Poh, for sharing with us today.
Thanks, Basil. Thanks for inviting me to share my thoughts on AIRI as well. It’s my pleasure to share with the audience the stories and thought process behind AIRI. I would just like to highlight again that if any company wants to know more about AIRI, wants to know more about how to apply AIRI to their organisations, to help them in their AI digital transformation journey, they can always reach out to AI Singapore. The AI Advisory team where I am from will have a more detailed discussion with them. Thanks, Basil. Thanks, everyone, for listening. Thanks for your time.
[*] This conversation was transcribed using Speech Lab. The transcript has been edited for length and clarity.
