AI - The random certainty

AI is transforming the development of software and the web. We look at the problems where the certainty of traditional programming meets AI built on probability.

How to stay in control

To me the major challenge I have had in programming for AI is the interface between the deterministic nature of programming up to now and the heuristic nature of AI and machine learning. When you program a computer to do "a" then you could expect to do "a". Because AI models and machine learning systems store their parameters in response to training without direct intervention from humans, they have to treated as probalistic machines where the output is not predictable. Of course there are controls and limits to what a an AI model will do, but at the macro level.

It may suprise lay-people that we programmers really don't know what is going on inside Large Language Models. While the principles are understood, the detail of how they arrive at their output is not. Programmers have had to produce rather strange tests to find out what is going on inside them. So, for example, Anthropic is making through training a test version of their models deliberately "obsessed" with the Golden Gate Bridge.

https://www.anthropic.com/news/golden-gate-claude

The idea behind this is to then trace the patterns of 'thoughts' within the parameters set and to see how well the teams at Anthropic can predict and therefore control the output. This is part of their tools for maintaining the safety of their normal "un-obsessed" models.

Although the pieces of programming with AI that I have done with have been small I can see that there is a problem with building a system which is in part predictable and is in other parts not. At the interfaces between these predictable and unpredictable elements it is difficult to control.

When this is scaled to large systems with some kind of agentic component there are a lot of interfaces between deterministic and heuristic elements. The problems are not only of keeping the whole system in control, but also of the programmer understanding how the whole thing works together when they are putting together where some things are determinstic and other parts of random.

It is amazing how ingrained the habits of dealing with the deterministic nature of computers affects your habits as a programmer in good ways and bad ways. The fundamental way to fix things that are broken in computers, debugging, is to isolate the problem behaviour and reproduce it, make it happen again. If you cannot reproduce the problem then it is not a problem. When you are working with a system that has a heuristic component then you cannot be sure that the problem will happen again, even if you have isolated the problem behaviour. You can only say that it is likely to happen again, and you can only say that it is likely to be fixed when you have done something to fix it. This is a fundamental change in the way we have to think about programming and debugging.

But AI as both a tool and a partner in programming is changing the landscape of what we can achieve.