Migrating a 200,000 LOC codebase to Typescript in a day and a half

About the author: Jacob Klapwijk is a Senior Frontend Engineer in the Presentation Experience (PX) Team and a Line Manager at Mentimeter.

A couple of years ago, when statically typing Javascript started to become popular, there were 2 main options to do this: Flow and Typescript. At the time they had a similar adoption rate, and both had their own pros and cons. At Mentimeter we chose Flow. Mostly because it was better at incremental adoption (we already had a big untyped codebase) and it would've been much more difficult to adapt it to Typescript. Another reason was that @Babel/typescript didn't exist yet, meaning that adopting Typescript would require using a different transpiler than the one we already used. This would have increased risk as well as a time investment.

Fast forward a few years: Typescript gets 30 times more downloads than Flow, it's tightly integrated with Babel, and it's slowly becoming a standard in the frontend community. Our codebase is a lot more mature, to a great degree thanks to having typed it with Flow. We don't see any reasons to migrate to Typescript at this point: most of our frontend engineers believe that we'd probably use Typescript if we'd start from scratch now, but we're too invested in Flow at this point. Migrating would be a massive investment and won't make a big difference - our code is typechecked already, right?

At that point, my colleague Jakob, who had extensive experience with Typescript, started. Within a week or so after onboarding, he started to pop questions into Slack. "How does automatic import work with Flow?" "Can anyone get automatic refactoring working in VS Code?" "Does Flow also crash for you from time to time?" We couldn't really answer these questions as we hadn't thought that much about them before. We might've heard of some of these Typescript features, but we hadn't experienced them enough to truly value them. I'll hand it over to Jakob now to explain how he handled a potential migration.

Getting the migration going

When starting to work with Flow, I quickly realised how much better the developer experience was with Typescript. I became frustrated and felt my energy was spent too much on fighting Flow instead of actually building features. I could see a lot of improvements to be won but I realised that a potential migration would be a question of cost versus benefit. I needed to find a way to decrease cost (time spent on the migration), and simultaneously convince fellow engineers of the benefit (increased efficiency and Developer Experience). The latter proved quite easy - a presentation of Typescript benefits such as editor integration, stability and community adoption took care of that. Getting the migration time as low as possible was now key.I considered some different approaches to the migration and came to the conclusion that a big-bang instant migration would be preferred. Flow and Typescript aren't cross-compatible, which would mean that a gradual implementation would drastically reduce the type safety of our code base. On top of that, their editor integrations are generally not able to live side-by-side in the same application, meaning that developer experience would be way worse during the migration time.

Doing a big migration in a short time frame is a recipe for disaster: there are too many chances to make a mistake, especially when you're working towards a deadline. Luckily for us, there is a codemod called flow-to-ts, which allows you to automatically convert Flow code to Typescript. It doesn't cover all use cases, but it would bring us quite far. With that in mind, I came up with the following process that would cover each repo:

• Install new dependencies: yarn add --dev typescript @babel/preset-typescript @types/react @types/react-dom
• Add a tsconfig.json file to the root directory
• Replace @babel/preset-flow with @babel/preset-typescript in any babel config
• Run the following command to convert the codebase, and save all conversion errors in a text file: npx --package=@khanacademy/flow-to-ts flow-to-ts --write --delete-source YOUR_GLOB_PATTERN 2>&1 | tee conversion-errors.txt
• Run prettier on the output. If it errors, it means that the previous command caused syntax errors - fix those before continuing.
• Go through the files that were outputted to conversions-errors.txt and convert them manually.
• Run npx ts-migrate reignore . in order to disable any Typescript errors that have popped up - we can fix those incrementally later.
• Run yarn tsc in order to typecheck the code base - fix any potential errors that are still left.
• Modify the Webpack config so it includes .ts and .tsx files to get the build to work
• Remove any Flow leftovers: dependencies, comments, the flow-typed directory
• Done!

In order to avoid merge conflicts & low type coverage during the migration, we decided to do this in a single push. We normally have a deploy freeze on Fridays after lunch, which we moved to Thursday: this gave us a 1.5-day window to execute the migration. In the week ahead we encouraged all developers to have as few as possible outstanding branches at that time: they might be painful to merge after the migration.

We put together a project team of about six developers with a voice chat going on Discord. First, we collaborated on running the script on all modules, going through the manual steps and releasing new versions of them on (private) NPM. After that, we split into groups of two, with each taking ownership of a single application.

In this process of making applications merge and deploy ready, we didn't have a lot of focus on type safety - it'd be too much work to try to fix all the type errors that Typescript picks up which Flow hadn't before. Instead, we focused on getting the application runnable, which is surprisingly easy. Theoretically, the runtime code should be identical, since Flow and Typescript both compile to Javascript.

In reality, there are some manual adjustments to do though. Mostly related to the build process. Doing this took about a day, which meant that by the end of Friday all applications were ready to release. Even though we didn't do any changes to the compiled code, we decided to wait until Monday to release the changes, to be on the safe side. This turned out to be a good idea: a small bug made its way through anyway. But considering that we rewrote 200,000 lines of code in 1.5-days that seems like an acceptable result.

Conclusion & Learnings

A couple of months later, we can conclude that migration was a great success. Engineers are very happy with the improved developer experience and we have the same, or possibly better, type safety as before. On top of that, new engineers joining us are more likely to have experience with the tooling we use.

A major takeaway from this migration is that it's possible to do big bang migrations safely, even if it is not preferred. If it seems complex, script it: this gives you the ability to dry-run the migration as many times as you want, streamlining the process along the way.

But most of all, we once again realised that developer experience is worth investing in. Not only because it makes engineers happy, but also because it enables them to build higher quality products faster.

If you find work like this interesting then maybe there might be a spot for you on our team. Check out the jobs page to apply for an open position, or contact us directly if you want to have a chat