Most people, if you asked them, would agree that Proper Planning Prevents Poor Performance. If you undertake a project, it needs to have a plan setting out what it aims to do and how it aims to do it. But knowing that a plan is critical to success and actually making that plan seem to be two different things altogether.
I recently read a famous IEEE article from 2005 called Why Software Fails, and one of the top reasons across a vast slew of major software project failures was “badly defined system requirements”. Technically worded like that, it sounds almost reasonable, as if it were just something that kinda happens, y'know. “Everything would have been fine, but we didn’t define the system requirements well.”
Huh?
Badly defined system requirements is corporate speak for having no real idea what you set out to do. You may have had a goal, even a realistic one, but you just missed out on the minor detail of planning how to achieve that goal. Just bad luck, I suppose. Next time we’ll get there.
I recently failed at my project of becoming a billionaire by the end of June — nothing catastrophic went wrong, I just didn’t plan how I was going to make up the almost billion dollar gap between my current and hoped-for wealth. Otherwise, all good.
The reason I’m writing about all of this is because I’ve experienced it a number of times as a freelancer building software and spreadsheet solutions for clients. I previously wrote about turning down projects that didn’t make sense, mostly due to spreadsheets being jammed in places that they shouldn’t, but the same thing applies in the rest of the software world.
It’s frustrating knowing that there are businesspeople with genuine problems that could be solved with elegant software solutions, but that the project doesn’t work out. More than the fact that money is being left on the table, it’s the fact that this situation is so preventable. The businessperson leaves the interaction feeling disappointed that they didn’t get somewhere concrete, and I add another brick in the wall of potential projects that never really took off. I’m happy that the project got scrapped as early as it did, before too much time, effort, and money got wasted, but when it happens repeatedly, the disenchantment grows.
For me, it usually goes something like this. I get a call from a businessperson looking to make something to help solve a process in their business. They’ve successfully identified that their current system isn’t doing what it should, or is requiring too much manual and repetitive work. (Since the birth of the LLM, the pre-existing appetite for automation and the relentless elimination of repetitive digital work has become insatiable. The alarm bell that goes off in someone’s head now when they realise they’re doing routine work is a good thing in principle — perhaps it’s gone a bit far and some non-technical folk are starting to fall prey to a disease previously only affecting developers — but that’s not the focus of this article.)
So I talk with the businessperson for around 30 minutes while they show me their current system and explain what’s wrong with it. Unknowingly, they’ve used a fair bit of domain-specific jargon that requires unpacking. These are shortcut words that their coworkers or industry understand but that they’ve forgotten other people completely do not. And this accidental tunnel vision extends from their language to their systems, where sometimes they implicitly believe that their workflow is a well-known one and of course I must know about it and know of a well-known solution to it. Once again, this is a good thought in principle, as many business problems are already solved and don’t require new wheels to be invented, but the devil is often in the details. Lord Varys once said that “a small man can cast a very large shadow”, and with software projects, sometimes a small, as-yet-unknown issue will completely throw off a solution and require an entirely new direction to be taken.
The breaking point then comes when the topic of the call turns to costs and time estimations. The IEEE article from above says “estimating how much an IT project will cost and how long it will take is as much art as science”, and this is as true nowadays as it was in 2005. It’s an inherently hard problem. But there’s also no issue with the potential client wanting to know what it would cost. The issue is that we’ve only spoken for at best 25 minutes, and it’s practically impossible for me to say much of real value at that point. Usually, I can tell fairly quickly if something will definitely not work. But most of the time, we’re in the middle zone where I could think of several potential solutions, but without knowing a whole lot more about the current system and the way the person works, I really can’t say for sure.
So I say as much to the person, something along the lines of “I can think of a few ways this might work, but can’t say yet what would be involved until I know more, so I can’t really quote you yet”. The discovery call is just level 1, an overview of the what. But still just an overview.
Once I mention that I think we need more time going over some details and getting into the thick of it, say another hour call, they start to get hesitant. The idea that the solution can’t be magicked away doesn’t fit their expectations. Details are for the developer to worry about, right? I’m just the business guy. I tell you what I need, I pay you, and you go do it.
If only it were that simple. It seems to me that some businesspeople simply do not want to spend the time getting deep with a pen and paper, really defining the current state of their system and how they want things to work in their new system. Of course, the developer should provide some creative insight here, using their technical knowledge as a basis for their answers, but if the businessperson fully farms out the thinking to the developer, then there will be no agreed definition of done, or more likely, there will be a neverending cycle of changes and revisions to the system the developer makes. It’s much easier to comment on issues with something that exists than to plan something better.
And I get where the businessperson is coming from. Their daily focus is their business as a whole, making money, serving customers, and so on. The processes they put in place to do that can feel like a necessary evil. They don’t want to get bogged down in the weeds planning out a software project. But if this is you, then you need to be realistic. There’s no magic way for a freelancer to siphon the knowledge from your head and understand your business to the point that they can successfully deliver a project. So if you don’t want to put in the planning, then be honest with yourself that you’re going to have to live with your problematic systems and you’re okay with that.
Sometimes, however, something remarkable happens, when a certain amount of humility is accepted, and between myself and the businessperson, we really get somewhere. I spend an hour hammering them with questions about this detail or that, and they take the time to think and answer properly. When this ends up crystallising the problem and potential solution, it’s a great feeling for both of us. One previous client actually told me how much they appreciated me forcing them to answer a number of questions, as it helped them clarify their entire business’s vision. “Clarifying your business vision” isn’t a typical selling point of software projects (except maybe for AI grifters), but you never know what will come out of some old-fashioned deep thinking.
In the last couple years, the dysfunctional version of the interaction has spread from businessperson–freelancer to businessperson–LLM. The AI figureheads are endlessly selling the dream of non-technical people “shipping” internal tools and custom software that solves all their problems. So it’s no wonder that people’s first reaction when they face a business problem is to turn to Claude or ChatGPT and try to get something going. Putting aside some inherent technical issues that will arise with this kind of thinking, I believe it only exacerbates the “badly defined system requirements” issue. Spending a few hours with a model creating a solution may give the illusion of progress, but without your deep thinking, the model can only go so far. It can prompt you with generally known questions you need to answer about the solution, but the niche domain-specific things that only you know will remain unresolved.
I’ll admit that this sometimes happens to me too as a developer. Instead of planning something properly, I’ll take the lazy route and start working on it with an LLM. At some point, I realise that I am working so hard fixing the thing it made to fit what I want, that I would have been better off spending a bit more upfront time defining my requirements. I feel ashamed that I took the short-long way when I know better.
This is something my English teacher was relentless about in high school. If you don’t plan a few bullet points for your essay, you may make a couple paragraphs of progress out of the gate, but will inevitably get lost and start to meander. Everyone knows the famous joke about how short it takes to prepare a long, spontaneous speech, but that a short and succinct speech needs a lot of planning. Opinions differ on whether the original quote is from Churchill, Mark Twain, or Woodrow Wilson.
All of this is not to say that everything must be scoped out in advance. To attempt such is fruitless. There will always be unknown unknowns, things you don’t know that you don’t know. The key is being honest and accepting that these will arise. A freelancer needs to bake in some measure of time and money for the existence of unknown unknowns, but this is a far cry from being able to solve a problem after being introduced to it in a completely unknown domain only 30 minutes ago.
So what should a businessperson do to increase the chances of their software project actually succeeding?
-
Think about the accidental jargon and industry-speak you commonly use, and define these while talking
-
Assume the freelancer knows nothing about your business, and give more detail rather than less. Let the freelancer pull you up short if you’re giving too much detail
-
Plan for the call by writing down some bullet points about the issue. Forcing it to be written down helps clarify vague concepts
-
Shift your expectation of freelancer magic from “they will know exactly what I want without me telling them” to “they will create a brilliant solution”
Spending some time in deep thinking mode (the human version, not the LLM feature) will yield strong positive results. You’ll know exactly what problem you’re facing, and the freelancer you engage will be pleasantly surprised that you’ve given them something clear to work with. They’ll be positively motivated to work with you, rather than looking at you as just another person who wants their problems to magically disappear without putting in some investment.
When you realise that the requirements are indeed — as the name implies — required, you’re giving the project a fighting chance of success, and in the software world, that’s as good as it gets. Software is complex and there are infinite ways things can go wrong. Don’t let “badly defined system requirements” stop you before you even begin.