POC: Switch to pnpm workspaces

[navDhammu]

Blocked by #3528

Description

This is a proof of concept PR which came up during the discussion of backend typescript PR. It attempts to provide a long term solution to some dev experience issues, such as making it easier to work with shared dependencies like typescript.

Currently we're using both npm and pnpm and this isn't ideal because they both have their own way of handling dependencies. Some problems with this setup that I can think of:

• Its harder to debug issues when they come up (and they always do) because you don't know if its caused by npm or pnpm or something else.
• Need to configure CI/CD and docker differently for npm and pnpm.
• Have to duplicate some shared dependencies, this is inefficient both in terms of disk space usage as well as maintainability.
• More confusing and error prone for new devs and contributors, making it more likely for them to unintentionally cause bugs that could have easily been avoided.

Proposal

Use pnpm as the sole package manager

Why?

1. It has built in support for workspaces. Workspaces is designed to work well with monorepos and takes care of things like shared dependencies out of the box. Npm also introduced workspaces but its not as feature rich as pnpm.
2. Speed - pnpm installs things faster and more efficiently (see chart below, source: https://pnpm.io/benchmarks)
3. Easy transition since we're already using it in webapp and everyone is familiar with its commands

Some drawbacks:

• It needs to be installed separately, unlike npm which comes with node.js.
• Some external tools may require extra steps to configure working with pnpm.
• Is newer, so less guides and stackoverflow answers for troubleshooting problems.

What is done

• Lerna is removed and replaced with pnpm workspace commands
• All package-lock files are removed, and replaced with a single pnpm-lock file in the root
• Npm commands are replaced with their equivalent pnpm commands
• Github workflows are updated to use pnpm
• TypeScript is installed in the root and is now accessible to any sub package

What needs to be done

• Docker files need to be updated
• Blocked by https://lite-farm.atlassian.net/browse/LF-4488

How to setup

1. Make sure you're using node version 20.18, and pnpm v9 or above. This is strictly enforced using engines and engine-strict options in package-json and .npmrc. Thanks to the workspace feature this only needs to be specified in the root and applies to every sub-package.
2. Remove existing node_modules from root as well as all packages. A shortcut command for this: pnpm -r exec rm -rf node_modules
3. Run pnpm i from anywhere inside the project, doesn't have to be root. This will install deps for all packages.