AI is the new compiler

What I think is one of the most interesting trends of this moment goes unnoticed by the general public, but surprisingly also by the majority of software professionals. Now that the latest AI winter is over (there have been a few in the past) and every self-respecting information system has at least a bit of AI in its bowels, the way we design and implement information systems is drastically changing. Instead of programmers writing down exact instructions that a computer must execute, machine learning specialists specify and modify both the input and output (I/O) of the system and leave it to AI to find the algorithm that does so on a training and testing set of data. (As a side note, the latter is important to realise, the exact algorithm is very hard to understand and so are predictions of it on unknown input data.) AI thus compiles the requirements into an executable algorithm.

I have been interested in ways we could improve the way we design and implement information systems for a long time. It is my belief that the current state of the art in software engineering is temporary and that we must move on to improve our profession for economic, safety, and societal reasons. Our current way of working is very cumbersome and leads to a lot of problems while at the same time software is controlling an ever increasing part of our daily lives. There is a long and very interesting history of software development from the 1940's up to the present day that I have been following for many years. If there is one continuous thread I would say that it's the fact that the software engineering profession keeps moving towards higher levels of abstraction. From machine code to assembly to C to C++ to Java to 'scripting' languages. But it seems the industry got stuck somewhere around the beginning of the 1990's. When the industry exploded with the advent of the web, so did the number of tools, but none of them was at a fundamental higher abstraction level than languages such as C. So what we got was basically more of the same. Unfortunately 'the same' turned out to be not good enough for our fast changing world. We simply don't have enough software developers to keep up with global demand while at the same time the stakes are rising for every piece of new software that gets released. Admitted, there has been research into more 'semantic' programming languages, but none of them left the academic realm to conquer the software industry.
With the advent of AI something interesting happened though. As said traditional software engineers writing down machine instructions are slowly being replaced by machine learning specialists selecting the right estimator and specifying the I/O. The latter work on higher semantic level, they are concerned with the properties they want the algorithm (system) to have, not about the instructions the machine should execute. This is a fundamental difference that fits neatly with the long historical trend of the software profession moving towards a higher semantic level. Of course the machine learning approach does not fit every use case, there will still be many systems that must be specified procedurally (or functionally if you prefer so), but the variety of use cases for machine learning surprised me (even as an AI veteran).
With machine learning specialist being the most sought after profession at the moment it is my guess that this trend is just picking up steam and we're only starting to scratch the surface of what I believe to be a fundamental change for the software industry at large.

Leave a Reply

Your email address will not be published. Required fields are marked *