feat: initial work to separate task processor from main repository

[matthewelwell]

In order to use the task processor in other private modules, we need to separate out the task processor module into it's own repository.

This will be added as a poetry dependency in the main repository (using git tags)

[zachaysan]

I didn't review the test code since I was already taking a while through the rest of it.

Most of these comments are more like questions or small suggestions, but I definitely think we should get the "make the tests pass" user experience better.

[zachaysan]

I'm wondering why this is necessary here but not in the main flagsmith repo manage.py.

[matthewelwell]

That's a good question. I think the reason stems from the fact that I actually pulled it from flagsmith-ldap instead. The reason it's different there (I think) is that flagsmith/flagsmith repo was created with django 2.x (because old) whereas flagsmith-ldap was created with django 3.x. I guess the actionable change here would be to update manage.py in the flagsmith/flagsmith repository rather than change anything here.

A quick test shows my assumption is correct:

❯ python -m django startproject testingmanagepy
❯ cd testingmanagepy
❯ cat manage.py
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys


def main():
    """Run administrative tasks."""
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'testingmanagepy.settings')
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == '__main__':
    main()

[zachaysan]

Sounds good. Maybe make a ticket so we don't forget to update the other one?

[matthewelwell]

I just made the PR instead :)

Flagsmith/flagsmith#3403

[zachaysan]

Should RUN_BY_PROCESSOR be in settings.py?

[matthewelwell]

I have a feeling there's a reason that this is done (perhaps because settings are loaded at the time we need the env var) and we copied it from somewhere in the django project itself I believe. @gagantrivedi may remember further details?

[gagantrivedi]

I'd argue that is not really a setting(in the global scope of things)
The use case, this:

       # The `register_recurring_task` decorator is executed in the global scope,
        # meaning it will run whenever the code is executed, including when running
        # `manage.py` commands such as `migrate` or `showmigrations`. However, this
        # decorator saves the task to the database, which is not desired unless it is
        # being run by processor, and to indicate the we set this `RUN_BY_PROCESSOR`
        # environment variable.
        
        ```

[zachaysan]

Missing typing here and below

[zachaysan]

Nit: missing return type here and below

[zachaysan]

I like this change 👍🏼

[zachaysan]

I reviewed the committed changes, not the whole thing again. Lots of nice fixes!

[zachaysan]

PR looks good!

[gagantrivedi]

Need to bump Postgres version

[gagantrivedi]

Should we add 3.12 as well?