If you've been keeping an eye on the AI landscape, you've probably come across OpenAI's ChatGPT Code interpreter which was released in Beta earlier this month. In the realm of data analytics, it's nothing short of impressive. With the capability to parse complex datasets and churn out meaningful insights autonomously, it's hard not to be in awe.
For example, I uploaded a dataset from kaggle on results from olympics in the past 120 years, and it was able to come up with summary statistics on top countries by medal count, distribution of medals, disciplines and more with 0 prompting effort. On prodding it a bit more, it was able to provide meaningful comparisons between countries and additional trends. All of this, with the working model of code in python and a set of visualizations that would take someone hours to days to do! It is fascinating how a language model that is primarily based on predicting the next character, is able to do really intelligent data analysis like this. It even goes in and creates custom metrics such as "medal efficiency" and describes it as the number of medals won divided by the number of events participated!
So here's what Code interpreter can already do: It can generate insightful reports and visualizations that could take a human analyst a significant amount of time to piece together. It can also run summary statistics, allow for a seamless conversation with your data and more. It autonomously debugs and rewrites code with no supervision and comes up with some conclusions. Watch this space for more - the pace at which these tools are being rolled out will change the way we look at data forever!
With such rapid advancements, it leads to the natural question - is the era of the human data team winding down?
I don't think so. I don’t see this as a countdown to obsolescence for our data experts. On the contrary, I believe it's opening the door to a new, transformative phase for data analytics.
Here are some reasons why:
1. Garbage In, Garbage Out
The long running saying of “garbage in, garbage out” will be amplified when it comes to AI driven data analysis. Having poor quality data, poorly developed data pipelines and unclear definitions of business metrics will directly impact the quality of analysis, whether it is human driven or AI-assisted.You still need to have your data pipelines in order - before you “hand off” your data to an AI to interpret, you have to make sure your data is accurate, clean and up to date. This is a set of tasks that requires specific domain and more importantly specific organizational knowledge. It may be time to start re-considering the approach of running a company on the basis of 35 excel files in someone’s computer!
2. Lower barriers to entry = More demand for data professionals
As data analysis becomes more accessible, more organizations will aim to become data driven. This will INCREASE the need for data analysts and not reduce it. With AI simplifying basic data analysis, more organizations are awakening to the strategic importance of being data-driven. Companies that once heavily relied on instinct or 'gut-feel' are now recognizing the value of empirical, data-supported decision-making. This shift will create a swell of demand for data professionals who can navigate the complexities of this transition. Moreover, as data-driven decision making permeates throughout an organization, there is an increased need for professionals who can provide the right training, support, and guidance to non-technical staff. The need to interpret and communicate the implications of data-driven insights, and to guide strategic and operational decision-making, is more important than ever.
3. Data Engineering & Analysis still requires significant human oversight
It's crucial to remember that deriving profound value from data isn't simply a matter of feeding information into an algorithm. The process requires a depth of understanding and a breadth of skills that, at least for now, are uniquely human attributes.
The process of transforming raw data into valuable insights entails not only technical expertise, such as in-depth knowledge of data modeling, data cleaning, ETL processes, and sophisticated analytical methodologies. It also involves a deep understanding of the specific business context in which this data operates. An AI might be able to identify patterns and trends, but understanding why these patterns exist, interpreting their implications within the specific organizational context, and then leveraging this knowledge to drive strategic business decisions, requires a human touch.
So what should data teams do?
For anyone working with data or running data teams today, there’s 3 approaches that you can take:
Shun these tools completely, and continue to work today as you have been for the past many years. [Hint- this is a recipe for disaster]
Embrace the power of tools like Code Interpreter and accelerate your data team’s productivity immediately.
Develop your own custom, AI-augmented data analysis tools by leveraging these LLMs and providing it with the context of your own organization’s data & use cases. [And if you are looking for help on this, reach out to us in the Newtuple team to assess]
I know I’m oversimplifying the options here because there are real considerations like data security and privacy, and even more nuanced ones like data quality. But if you take a step back, it really does seem like this is a moment in technology where the ones that adopt this technology will have exponential benefits over those that don’t.
Imagine a data analyst in your team that is able to leverage the power of these models today - s/he will be able to build analysis on data sets in maybe 1/4th the time it used to take earlier and also provide more in-depth work within this time. Your data team can immediately become more productive if they’re able to harness these tools in the right way.
The role of the data professional is evolving, not disappearing, and human oversight and expertise will continue to be paramount in the process. The combination of AI and human acumen, it seems, is set to be the gold standard for extracting the most value from our ever-growing volumes of data.
If you’re looking for advice on how best to leverage AI-assisted analytics in your organization, do reach out to us here. We’re already working with our clients to help them navigate the explosion of AI tools - there’s no one size fits all solution, some organizations may just be better off using off the shelf large language models, while for others there’s a real case to be made to develop an organization specific augmented AI data analytics tool, with all the bells and whistles of data security & if required even custom model deployment. Reach out to us here
Opmerkingen