Can ChatGPT do a CIO's job? Let's find out!

Published: 2023-06-19
Updated: 2023-06-19

The DevOps Dream is a simulator of a Chief Information Officer (the role you take) leading a variety of organizations through a digital transformation. It is not going to be an easy task, as getting the highest score possible relies on keeping your staff and customers happy, as well as ensuring stability of the product. The game was surprisingly accurate and reflected several situations I have encountered in reality. I encourage you to try it yourself - https://dream.devops.games/

This is out company logo!

This game was created by a consulting company based in Perth, Western Australia - Mechanical Rock. You can read more about how the game came to be here

I described the game scenario and the chatbot is eagerly awaiting for the next prompt

I fed it some background information, including the 20 initiatives available to the CIO to enable and off we went:

Here is was ChatGPT had recommended for the first year

Based on the provided list of initiatives, here are three key areas to consider for the first year:

“Automate application deployment” Option (c): “Choose several options for teams to select from” (Cost: $10 million). Automating application deployment is a crucial first step in streamlining operations and increasing deployment speed and reliability. Providing a variety of tools for teams to choose from can ensure that each team finds a tool that best fits their specific needs.

“DevOps Transformation” Option (c): “Choose a few blended teams to pilot DevOps” (Cost: $18.5 million). It’s critical to start nurturing a DevOps culture early on, as this transformation could take time. Starting with a few teams can provide valuable insights and learnings that can inform the broader implementation.

“Migrate to the Cloud” Option (b): “New systems in the cloud (cloud-first)” (Cost: $26.5 million). Moving to the cloud is a big step towards digital transformation. By adopting a “cloud-first” policy for new systems, Elephant Enterprises can start leveraging the benefits of the cloud immediately.

Unfortunately, it ran over budget by 5 million, which I had to remind it about. In its second iteration, the budgeting was a lot more conservative.

"Automate application deployment" Option (b): "Buy a COTS build & deployment tool for all teams" (Cost: $5 million). This option still provides the benefit of automated application deployment, but at a lower cost than the previous option.

“DevOps Transformation” Option (b): “Create temporary Dev and Ops teams for production issues” (Cost: $12.5 million). This option still begins the important work of transitioning towards a DevOps culture.

“Uplift skills” Option (b): “Centrally controlled training budget” (Cost: $5 million). Investing in skills development early on is crucial for a successful transformation.

Interestingly enough, the chat bot decided not to recommend migration to cloud anymore. Instead, we were now to invest into skill uplift among the staff. Let use see how the first year plays out.

Start of Year 1/3 with half of the budget spent.

First quarter is looking ok so far - we are trending upwards across 3/4 metrics.

Onwards and upwards!

Every quarter, there is an event that takes place and it requires a response from the player.

This is an example of a very difficult real-world question. Where to find your talent!

Choosing a talent strategy should align with the company's goals, culture, and the resources available. Here's a brief analysis of the three options:

…

Considering Elephant Enterprises is trying to reinvent itself digitally and needs to build capability to move faster, hiring only graduates might slow down the progress in the short-term. However, only targeting 10X developers might stretch the budget too much and create potential teamwork issues.

Therefore, I would recommend to Target the middle of the market. This would allow the company to bring in talent with some experience (thus not slowing down progress too much), while also ensuring that the hires are within the budget and promote a balanced team dynamic.

Given the limited information provided by the event - I found this response fairly adequate, despite a few assumptions ChatGPT is making. For example, we do not know what the staffing budget is and can only estimate it based on the limited information about the company as well as the project budget we were allocated. The response I gave when I was playing this game on my own was exactly the same. Let’s see how the metrics are affected by this decision:

Steady as she goes

We may have not gone down across the metrics, but we are not actively sinking yet. We are closing in on the first year and are about to face another set of initiatives that can be executed for Year 2.

For the second year, given your company's goal to become more nimble and innovative and considering the constraints of our remaining budget ($27.5 million), I'd recommend focusing on the following initiatives:

“Better requirements specification” - Agile user stories ($5 million). This approach is cost-effective and aligns well with the goals of increased speed and flexibility. Agile user stories put the user first, enhancing the overall customer experience.

“Invest in Code Maintainability Software” - Code quality tools ($10 million). High-quality code is easier to maintain, less prone to bugs, and can be updated more quickly, which aligns with your company’s objective of becoming more nimble and innovative.

“Automate testing” - Test automation center of excellence ($10 million). Automated testing will increase the speed of your development cycles and improve the reliability of your software by catching bugs earlier in the process.

The total cost for these initiatives is $25 million, leaving you with a budget of $2.5 million for unforeseen events or future investments.

Yes, indeed, we are only at the start of the second year of the transformation project and we are almost out of money! I am hoping that ChatGPT might realise that some of the initiatives can give us some money back and will consider taking those options in Year 3. For now, lets see how the year shapes up.

In the next event, the lead software engineer resigns. We have an option to hire someone with a similar skillset, promote from within, or, hire an developer from a successful local cloud startup. Over to you, ChatGPT:

Given the goal of becoming more nimble and innovative, I would recommend the following option:

Hire an experienced developer from a successful local cloud startup

This choice has several benefits:

It brings new perspectives and fresh ideas from an individual who has been part of a successful innovation in a similar industry. The new hire will likely have strong knowledge of modern practices, cloud technologies, and experience working in a fast-paced, innovative environment. This strategy can also help in creating a stronger competitive edge in the market, by integrating practices and technologies used by successful startups.

This decision is aligned with the transformation goal and can help foster a more innovative, adaptable, and modern culture within the software development team.

This manoeuvre appears to be a success! Mind you, the positive trend may also be attributed to the initiatives providing a positive impact. Unfortunately, there is no way to break down the trends down.

Oh, we happy.

In the 4th quarter of the year, we came across a peculiar event - “A key systems hardware has come to end of life. Your data centre is out of space, so you must move it to the cloud. What strategy will you take to re-host this system?”

Port the system to VMs running in the cloud
Rearchitect the system to be cloud native
Replatform the system to containers in the cloud

Given the details of the event, I would recommend the following option:

2) Port the system to VMs running in the cloud

This is also known as the “lift and shift” approach. It’s often the fastest and least disruptive way to migrate an existing system to the cloud. This approach minimizes the changes to the architecture of the system and reduces the risk of introducing new bugs or issues during the migration process.

This resulted in a negative trend towards the end of Year 2 and we can only hope that the effects are short-term.

Its a roller coaster!

At this point, I had to put this article away as I hit the dreaded “25 prompts every 3 hours” limit in ChatGPT. If you are reading this in the far future, I sure hope every human has access to ChatGPT now or it’s analogue, without any limits!

Regardless, we are down to our last 3 recommendations and off we go into year 3:

...

“Restructure software teams”: Project based teams with a support role (costs $1.5 million). This strategy will help you streamline your teams and enhance their productivity by organizing them around specific projects.
“Employee Assistance”: Setup an EAP hotline (costs $0 million). Even though it has no cost, setting up an Employee Assistance Program (EAP) hotline can be a significant booster for employee morale and well-being, which in turn could improve productivity.
“Automate infrastructure deployment”: Repeatable playbooks for operations (costs $0 million). Implementing playbooks can standardize your operations, reducing errors and increasing efficiency.

…

And here is the final result:

Did it do a good job? We might need to benchmark this game against a real CIO

It’s hard to say whether every CIO in the world needs to start looking for a new career yet. I would need to replay this game a few times and see how ChatGPT behaves over time. I am impressed wit the outcome, especially given that my first crack at the Elephant Enterprises scored very similarly.

Here is my try:

Give this game a go yourself too. It is a fun look inside the management of Software Development companies.