"How do you keep your software reliable?"
Ever wondered how we, at 1st Money, keep our software so reliable? In short, we do it by every day:
- Exactly matching all 8,200 of HMRC's model tax calculations,
- Publishing 272 product videos,
- Running over 6,000 click-navs,
- Checking over 30,000 asserts,
- And running over 1,000 delves.
The video's numbers are up-to-date as of the 14th of February, 2024. Keep watching, to see how it all works.
We test our tax calculations by inventing 15 different imaginary taxpayers, shown (at the top of the screen).
Each one has different settings. For example, how many hours the imaginary taxpayer works, how much is paid, whether the imaginary taxpayer's married, whether there's children, and much more. We use our 15 different imaginary taxpayers to exactly match all 8,200 of HMRC's model tax calculations.
Each tax calculation is matched to the penny, giving us recognition from HMRC that our tax software is 100% correct. And, of course, we sit down and re-verify them all with HMRC every year.
So far, we've made 272 tutorial videos, explaining every part of the 1st Money product.
Everywhere in the 1st Money product, you'll find video buttons . When you tap one, a video pops out from the side. In our help pages, almost every sentence can be tapped on to show a video explaining that step.
To make it all work perfectly, we've built our own video player, and video creation software. The videos mean that any customer that wants help can immediately find answers.
Questions and answers
We've written 2,000 questions and answers, explaining even more parts of the 1st Money product.
The questions and answers are searchable through an AI powered search box. When you type a question, the AI interprets your question's meaning, and then shows you the best matching answers.
It's fast, and it's accurate, and we're always adding more answers.
To make our software look beautiful on both laptops and smartphones, we've built our own library of 40 software widgets. Each widget has its own testing page, showing the different ways it can be used. The testing pages help us spot if any of the elements has any errors.
Every night, we use robots to do 10,000 taps through the 1st Money product on 6,000 pathways. The taps push our test server's 8-core CPU to 100% for over 9 hours straight.
On each pathway, a tiny robot taps a different path through the 1st Money product as fast as the CPU allows. It's like you're seeing on-screen right now. If the robot spots anything wrong, it takes some screenshots and writes some notes about the error. Using the error notes, our software engineers then make fixes.
By using click tests, bugs are usually spotted by us first, and not by our customers.
At 1st Money, all our software engineers use specialized engineering tools for their day-to-day programming.
Every time a software engineer finishes making changes to the software, the software engineer tools analyze the code the software engineer's written. The analysis warns the software engineer about errors like missing functions, unused variables, and much more.
Every software engineer, twice a day, runs thousands of asserts to test our software. The asserts include 10,000 tax asserts, and 19,000 system asserts.
Each dot you can see on the screen represents about twenty asserts. Altogether, the 30,000 asserts take about 1 minute to run. The asserts make sure that day-to-day software changes don't accidentally break our existing code.
Every software engineer, twice a day, runs 1,000 code checks, taking about 1 minute for the entire set. The code checks scan the 7,000 files in our codebase, looking for things to improve in the code, including:
- Code that's too complicated,
- Things that need to be named more clearly,
- And much more.
The delves mean that each day's changes are continuously scrubbed clean.
Every software engineer, twice a day, runs a spider that automatically opens and tests a sample of over 500 pages, spread across 1st Money's entire system. It takes about 1 minute each time, to test them all.
The spider helps make sure that obvious problems like dead links or page-load errors are caught by us first, and not by our customers.
Every software engineer, twice a day, runs a script that scans all our documentation and help, searching for sentences that can be improved. We check for things like:
- Sentences that're too long,
- And much more.
And that's it! That's everything you need to know about how we keep our software reliable at 1st Money. We've loved building it, and hope you love it too!
Was this page helpful?