How to scope work 🔬
A journey from the vague to the precise
👋 Hello!
Hope everyone’s doing well. It’s been a busy week at work this week, so this is coming out a little later than desired. I apologise for that.
If it’s your first time reading, here’s a little refresher. The goal of this newsletter is to discuss techniques for building better software and being more effective in high growth companies. I mix in longer form content — think a short blog post that you could read in 5 minutes — with some of my favourite links.
Over the last few weeks, I’ve covered:
I’m matching donations to Black Lives Matter causes up to a limit of £500. Please send me your receipts and I’ll match them straight away.
On with the show,
Stephen
How to scope work 🔬
As a software engineer, you’ll often be asked the following question: how hard would it be to do X?
Get used to it — it’s a very reasonable question! We’re here to talk about scoping: a process of answering that question.
What is scoping?
Scoping work is the process of going from the vague to the precise: from the idea of a feature, to a plan for how you might build it. It’s a broadly applicable skill — it isn’t unique to software engineering.
When you scope, the goal is to figure out and define what problem you’re trying to solve, and propose an idea for how you’ll solve it. The problems you’re trying to solve might be:
Adding a new button to your homepage
Building a new authentication system
Rolling out an engineering progression framework
This is not estimation. Estimation is giving an idea of how long something will take to build. You need a clear scope to be able to give that estimate.
The output of any scoping work I do is always a proposal. Want to know how to write good proposals? Refer back to my previous post on them!
Why is it useful?
Scoping helps you:
Align teams on the problem to be solved and the extent to which you’ll solve it in this version
Gain some level of confidence about what you have to do, and how long it will take
De-risk the process of building it
Scoping is half of the job of a software engineer. Your job is about translating things that people would like to exist, and figuring out how you make them real. You might think that’s what a product manager is for. Wrong — it’s your job.
How to scope
Decide what resolution you want to scope at
You can scope with different levels of detail in your scope. If we’re being really lazy, we can create two categories: vague, and precise. It’s helpful to start vague, get early feedback, and then go precise.
Vague, or ‘magic box scoping’
In this model of scoping, you have a bunch of boxes that you draw out that will perform some behaviour. You don’t know exactly how it will work yet, but you have a good idea that this is the right ‘shape’ of the system. You define the interface, not the implementation.
This technique is really useful when it might not be totally clear what the problem is you’re solving, or many team members may have differing opinions. It’s a helpful technique to make sure you get on the same page early on. As Keynes said, “it is better to be roughly right, than precisely wrong”.
Precise scoping
This mode of scoping defines both interface, and implementation. You’ll be specifying data structures, RPC call stacks, flow diagrams of requests, and so on.
The goal of this type of scope is that you should simply be able to sit down at a keyboard and type. There’s no unanswered questions.
⚠️ A word of caution. Do a ‘magic box’ scope, before you get here. Otherwise, you’re sinking effort into something that you might have to throw out later. Define the interface, agree upon it, then define the implementation.
Break the problem up into smaller chunks
Larger problems are hard to solve. Decompose the problem into smaller ones. Let’s take an example that’s dear to my heart: building a system for fraud detection. When viewed as on its own, it’s really hard to figure out how you’d scope out an approach to it. But think about it as a set of smaller problems:
Figuring out what data might be useful to have
Collecting that data from your users, and storing it
Building a system that can process that data and make estimations about likelihood of fraud
A dashboard to present those results to users
It’s much easier to figure out approaches to these sub-problems than it is to solve the whole. It’s somewhat like Elephant Carpaccio — sorry vegetarians.
Focus on high risk components
As mentioned earlier, scoping helps de-risk the process of building. This risk is not evenly spread amongst sub-components of your system. You need to focus more attention on the central components of your system which you understand poorly. Components that are high risk, and poorly scoped can often lead to misunderstanding and pain down the road.
Let’s go back to our food delivery service above. The highest risk box there is Stripe. Why? Well, we’re using an API that we’ve never used before. It involves sensitive data. We can’t really launch our service without charging for it either! It’d be well worth spending more time figuring out precisely what information you need to collect to charge a card, and how the interaction with your billing service will work. The user service is more likely to be well understood: an email address, name, and so on.
Figure out what parts of your system you can leave at low resolution, and which parts need high resolution scoping, and spend time accordingly.
Know the edge of your technical knowledge
This is one of my favourite comics. The irony is that this is totally a solved problem, given how good computer vision is getting.
Know the edge of your technical knowledge. Highlight where you don’t understand something particularly well, and where you can’t figure out the details.
Write a proposal
What I’ve described above is the thinking process behind any good proposal.
Define the problem to be solved, and figure how you’ll solve it. Then, take what you’ve figured out, write it down, and share it.
Things to watch out for 🙀
From experience, there are a few things that should set off your spidey senses of trying to get to a clear scope.
External third parties
Third parties can be amazing, or awful. One thing remains constant: you know less about the third parties’ business and way of working than you know about yours.
Assume that they are unlikely to operate at the same timescales that you do, and are unlikely to have incentives aligned such that they change to meet them.
De-risk early on by talking to them, and highlighting timeframes that you’re operating under. Try and to ensure that you aren’t in a situation where they can block parts of your process.
Scoping massive projects upfront
When a project is massive — e.g. a multi-year engineering initiative — scoping it becomes less effective. Uncertainties dominate. You’re not going to do whatever was in your plan. In these situations, do a rough scope, and sequence work by time. As you progress along timelines, scope future work in more detail.
Best of the internet 🔗
Every week, I collate some of my favourite links to share. Found something cool, or built something great? Send it to me by replying to this email and I might include it in the next edition. 📧
Why software projects take longer than you think: a statistical model - Erik Bernhardsson
This is very topical after what we’ve discussed today. Most projects take less time than you’d estimate but some things are far, far longer. Tasks with the most uncertainty dominate. Use your scoping skills to reduce this uncertainty where possible.
Growing the Internet Economy - John Collison
The Collison brothers at Stripe are exceptionally clear and structured thinkers. I really loved this podcast with John: a whistle-stop tour through payments, the internet economy, and product.
Performance reviews as a story - Sean J. Taylor
This was a remarkably useful thread about helping your manager write a great performance review for you. It’ll take you a minute to read, but might increase your earnings substantially over the long run.
That’s all from this week’s High Growth Engineering. If you enjoyed it, I’d really appreciate it if you could do one of the following:
Share it with a friend that would find it useful.
Follow me on Twitter: @sjwhitworth
Subscribe: just hit the button below.
All the best,
Stephen