I replaced Google with ChatGPT for most technical questions.

I tried ChatGPT4 for Unity, and it failed to understand complex problems that required systems integrations. Unity is a monster of integrated systems. It is as if you were to combine Photoshop, Illustrator, AutoCAD, Solidworks, and ProTools in one software. GPT-4 and GPT-3.5 can guide you in implementing and integrating something from scratch. However, since the AI cannot see your whole codebase, it can’t tell the entire context of your problem. You can give it pieces (even long ones) of your code and it does a good job explaining every single line of code. But it often fails at providing a solution when it doesn’t know what your other dozens of files have.

ChatGPT wouldn’t replace (some) humans in the future. Let’s say that a future version of ChatGPT can plug into your whole codebase. It might be able to fix bugs, refactor code, and even write new code. However, it cannot interface with non-technical humans to understand the context and intention of a problem.

If non-technical stakeholders requested a report to ChatGPT to integrate data and get insights for a business segment, it would just output instructions and the code to build it. However, this is where ChatGPT would fail:

  • It might provide complex code unless they request “easy to understand” code.
  • They wouldn’t know what to do with the code. The AI is not smart enough to log in to all the required systems to implement the solution.
  • Let’s say they give the code to a (human) engineer who tries to implement the code but finds no clear association among the underlying database tables.
  • The engineer goes back to ChatGPT to explain the problem. Not sure if the AI can provide output to the following: You give it the table schemas and sample data, and you expect it will tell you how the tables are associated.
  • You would give ChatGPT the DDL codes to tell you what you already know. The code has constraints and associations with other tables. However, in this particular case, even the DDL doesn’t show there is an association between table A and table B, even though you can see that the UI shows that there is a relationship among data from both columns.
  • Can you give the AI a screenshot of the UI? They said the next version of ChatGPT4.X can generate images. But can it understand screenshots of anything? When you paste an image containing text in a Word (or was it OneNote?) document, it recognizes the text if you search for it.

ChatGPT as an assistant

I use Google to search for answers, and navigate through the pages to see if I can find the answer. Most of them are often from StackOverflow. I wonder why I don’t go to StackOverflow directly. Sometimes, results are blog posts loaded with ads. Rarely do I end up in someone’s blog where they care more about the content than with annoying ads. I also use Youtube to find short tutorials. However, I must watch unnecessary content to get to the actual answer.

How is ChatGPT replacing this experience?

It provides context based on what you previously searched for.

Here are prompts for a conversation with ChatGPT. I was having problems animating a door in Unity. I had an idea what a Prefab was but wanted more details.

Me: What is a Prefab GPT: …answers in detail

Me: I have a Prefab door that is closed. When the character gets near, the door opens. When the character crosses the door, the door closes. How would you use Mecanim to do this? GPT: …answers in detail, step by step (with steps in bolded sections), with sample code

Me: This code doesn’t work GPT: …answers in detail

This conversation went on and on back and forth. However, after each prompt I didn’t have to provide the whole context. It un