We can’t accurately predict AI’s impact but we can explore some possible disruption points.

Source Code:

• Currently most software systems are developed using source code – a human readable, (hopefully) deterministic specification of system behavior. Development tools that allow end-user programming often bypass the need for source code by using record/playback and visual programming-style interaction modes. When behavior needs to be altered, a source code representation is useful. It aids system understanding and enables fine-grained, parametric control[1] of behavior.

• There is a broad class of systems for which most behavior is determined by variation in data. This class includes traditional database systems as well as ML and NN-based systems which create internal (usu. non-human readable) representations that drive behavior. As useful as these systems are, they are not transparent or easily modifiable in a parametric way. We sacrifice the rapid, fine-grained control of behavior that source code provides in order to do things not easily achievable otherwise. Source code is a compact way of parameterizing behavior.

• Programming languages often build on top of each other. Compilers and translators for new programming languages often target other high level languages. When they do this, they are able to leverage existing development platforms and ecosystems. This bootstrapping process has the nice side effect of giving us a human readable representation (the generated source code) but in most cases the build process pipeline eventually collapses, removing the use of source as an intermediate language. Example: C++ was originally implemented as a preprocessor for C. Eventually, C++ compilers compiled directly to lower-level representations, object code etc.

• Today, GPT-based systems are being used to produce source code as an intermediate representation. We review and then feed that generated code into the build process. It’s worth considering whether this is necessary. The near term answer is: maybe. The output of GPT-based systems is probability-based, alterable by a “temperature” that produces novelty in solution generation but also the chance that a solution may “miss”, be wrong, or have bad consequences (the “hallucination problem”). Human validation is currently a key part of the process and source code is a traceable medium.

• As development moves to a generate-and-test paradigm, quality assurance will have renewed importance.

• There is a strong possibility that prompting replaces source code to some extent, however, ‘temperature’ is a problem. To the degree that AI produces novelty, prompts are no longer deterministic specifications of behavior. Regardless, we may see the development of prompting languages with the aim of specifying must-have behaviors and placing constraints on stochasticity. It is likely these will be developed ad hoc in GPT sessions and later standardized.

Modularity:

• Will systems continue to be built from modular pieces? Let’s look at two common ways of composing systems from parts. The first is gluing pieces together. The other is creating pieces with ‘holes’ or extension points. In loose terms, the former is the library composition approach and the latter is the framework approach. We can view pieces ‘with and without holes' as modules.

• As the sizes of AI context windows increase, the size of generated modules can increase. These modules can be read by AI, modified through prompts and re-generated on demand.

• Most systems that generate code have something I call the ‘escape hatch’ problem: Invariably there are things can not be easily done with the generator. Traditionally, the way to handle this has been to provide a way to ‘drop down’ into a lower level representation and fill in the details. To the degree that this problem persists, generated modules can provide ‘holes’ as ‘escape hatches.’

• AI enables a third way of composing software. A module with extensive functionality can be read by an AI and then rewritten with only the subset of functionality and any additional tailoring needed for a particular use. This is like an automated private fork. High value code bases can be reused in this way.

• Another possible scenario for development is the use of peer AI systems that negotiate feature sets with each other in order to arrive at a system design. It is hard to know what this means for modularity but it is likely that Conway's Law will hold in many cases with the modularization mirroring the interactions of the peer systems responsible for producing the design.

Closing Thoughts:

• Source code seems to have value for the immediate future. We could arrive at trained AI that, through interaction and learning, “understand” a domain and its constraints so well that they can generate large target systems on request. If that happens, it might make sense to simply persist context as a set of embeddings, bypassing the need for source code.

• The relatively small size of context windows that we have today seems like it would be the primary motivator for the continued use of modularity in software development, but all of the uses of modularity outlined by Baldwin and Clark[2] remain. Modularity isn’t just a software concept; it’s a strategy used in nearly all systems of scale.

This is just the way things appear to me now. It will be interesting to see how it all plays out in the near and long term.

[1] Parametric control - a system behavior is parametrically controllable to the degree that it can be changed concisely and deterministically with specificity.

[2] Baldwin and Clark - Design Rules: The Power of Modularity

No AI were harmed in the writing of this article.