Add a Comment   Up to OJB's Blog List

Why is Programming Hard?

Entry 2069, on 2020-08-11 at 15:34:42 (Rating 1, Computers)

Why is programming so hard? It's a question often asked by some people, but others say it isn't hard, it's easy. We often see non-specialists doing successful programming which might be quite complex, but on the other hand, many professional programmers - even those working for the leading software companies of the world - seem to have trouble creating solid, reliable, fast, and elegant programs.

So do amateurs do a better job than professionals? It is tempting to say "yes", but I doubt whether that is true. The software created by amateurs (and I use that term based on it not being their actual profession rather than them being necessarily less proficient in some way) tends to be only used by them, to be only used on one computer type, and to perform a single task, although it is often a quite esoteric, complex task. A classic case would be some sort of analysis of scientific data, which sounds a lot harder than word processing (for example), but in fact isn't.

But even if we accept that the professionals are better than amateurs we still have to ask why the software produced by those professionals is so much more problematic than the items produced by professionals in other areas. For example, computer software fails a lot more often than computer hardware, or electronics like TVs, or even cars. Why?

First, let me explain why programming might be difficult. It is because there are so many fine details to be accounted for, and it is very easy to overlook one small item which might cause catastrophic failure. As an analogy, image you are writing a lengthy novel (War and Peace or Atlas Shrugged, for example) which are about 560,000 words. That is equivalent to a medium sized software project, which gives the scale of what is being attempted.

But in the novel a small error, like a mis-spelling of a character's name or a minor inconsistency in the history of a character's life would probably not even be noticed. In a software project calling a character (or in the programming context, a variable, or method, or function) something slightly different (let's say "bob" instead of "Bob") might cause a crash, or worse still: not a crash but an incorrect result.

And those two uses of Bob's name might be 1000 pages apart. And one might be written several months after the other. And they might be written by two different programmers, if a team is involved (which it will be for most medium sized and all large projects). That makes errors a lot more likely.

And if that wasn't bad enough, using a program is a bi-directional phenomenon. The program provides material to the user, but the user provides input to the program too. It's like an interactive novel where the reader controls the flow of the story in many ways, all of which must be handled perfectly.

But wait, there's more. A novel is always "used" in a controlled environment. The person might be reading a traditional book, or maybe reading an electronic book, or listening to an audio book, but that's about as far as it goes. A program is asked to run on many different computers, with different operating systems, and to interact with whatever other programs might already be there. That makes everything far more complex and error prone.

So having partly excused the problems with programs, let's look at a particularly significant example of failure. Modern aircraft are very complex, and there is potential for problems with many of their systems: control systems, communications, the strength of major components, etc. But let's look at the most famous example of a recent aviation failure: the Boeing 737 MAX 8 debacle. Two crashes of that aircraft - and the resulting crash in Boeing's sales - were caused by software errors, not failures in any of the other extremely complex components.

The problem there was at least partly due to another common problem with software production: poor management decisions. Of course, this is a potential problem with any area of engineering, but software engineering seems to be more susceptible to it than most others. In the Boeing case, the software production was reputedly outsourced to cheap programmers from India. There are many great programmers in India, but they are unlikely to be working for a company bidding for the lowest price from another country, so that would seem like an accident waiting to happen.

So programming really does seem to be more challenging than it should be. What is the answer? Maybe we need a new paradigm. But what is it? At this point, I usually jump in with a brilliant suggestion, but not this time. However, I will make a few simple suggestions which might help in some cases.

First, programmers should be paranoid. They say you can tell a programmer crossing a one-way street because he looks both ways. That is the type of caution required. Extra checks are required, even when a previous check might have made them apparently unnecessary.

Second, keep it simple. it is very tempting to write incredibly obscure and compact code just because you can. But writing 5 lines of clear code is better than one line of arcane code. And modern compilers are good at creating efficient output, even when the source might be more wordy. I sometimes look back at what I have done in the past and wonder what it does; then after looking at it for a while I think "wow, that's pretty cool", but maybe it would be better if I just understood what it did straight away!

Third, avoid using code written by other people. Avoid libraries of functionality from other companies. And avoid copying and pasting code from other projects. It is just too easy to make assumptions about how other code components work, or to miss a subtlety in a library, or to not understand something copied from another source. When I "borrow" small code components, I tend to look at them and re-write them in my own style. It takes a bit longer but it avoids many errors. Of course, that is often impractical, but I do it when I can.

Fourth, test programs using the most ridiculous options and user interactions. Better still, ask a typical user to use the program and note anything they do that causes problems. If a user does something "strange" which causes an error it is the program's (that is, the programmer's) fault, not the user's.

Finally, try to keep managers away from any decisions they might not understand - and that is usually all of them. Technical decisions are better made by people with technical knowledge, and the cheapest option is almost always the worst. Boeing lost far more as a result of their decisions to opt for cheaper programmers than they would ever have gained through that decision.

None of these ideas will make software perfect, but they might improve the situation a little bit, at least until the new programming paradigm arrives, whatever that might be.

Comment 1 (5375) by Anonymous on 2020-08-14 at 16:09:29:

Programming isn't hard. I do it and I'm not even trained in IT.

Comment 2 (5376) by OJB on 2020-08-14 at 18:23:40:

Medicine isn't hard either. I make correct diagnoses all the time, with the help of "Doctor Google", and I could treat problems too, if the system would let me write prescriptions. My point is, that everyone can do everything to some extent. Programming is easy, but programming well is a lot harder.

Comment 3 (5386) by Anonymous on 2020-08-19 at 23:55:18:

Programming is easy. That's funny. Programming well isn't easy. Lots of people think they are programmers...

Comment 4 (5388) by OJB on 2020-08-21 at 12:21:35:

Yes, that is exactly my point. Everything is easy if you don't want to do it particularly well. Some things are even fairly easy to do to a high standard. Programming isn't one of those; it's easy to do badly, but hard to do well.

You can leave comments about this entry using this form.

Enter your name (optional):
Enter your email address (optional):
Enter the number shown here:
Enter the comment:

To add a comment: enter a name and email (optional), type the number shown, enter a comment, click Add.
Note that you can leave the name blank if you want to remain anonymous.
Enter your email address to receive notifications of replies and updates to this entry.
The comment should appear immediately because the authorisation system is currently inactive.