Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flet new packages structure and flet build v2 #4122

Merged
merged 100 commits into from
Oct 9, 2024
Merged

Conversation

FeodorFitsner
Copy link
Contributor

@FeodorFitsner FeodorFitsner commented Oct 8, 2024

Closes #3839
Closes #3050

Summary by Sourcery

Revamp the Flet CLI with a new package structure and introduce version 2 of the flet build command. Add cross-platform permissions, support for compiling Python files, and deep linking configuration. Enhance the build and CI processes to support multiple platforms and Python 3.12, and improve security utilities. Update documentation with new README files for Flet packages.

New Features:

  • Introduce cross-platform permissions for iOS, Android, and macOS apps, allowing configuration of location, camera, microphone, and photo library access.
  • Add support for compiling Python app and site packages to .pyc files with options for cleanup after compilation.
  • Implement deep linking URL configuration for iOS and Android builds.
  • Introduce new command-line arguments for specifying architecture, splitting APKs per ABI, and Android signing configurations.

Enhancements:

  • Refactor the Flet CLI to use a new package structure, improving modularity and maintainability.
  • Update the build process to support Python 3.12 and enhance the AppVeyor CI configuration for better platform-specific builds.
  • Improve the handling of platform-specific dependencies in Python packages, including the introduction of a script to update wheel dependencies.
  • Enhance the security module by implementing encryption and decryption utilities using the cryptography package.

Build:

  • Update the AppVeyor configuration to build Flet packages for multiple platforms, including Windows, macOS, Linux, and web.
  • Add scripts for patching Python package versions and names during the build process.

CI:

  • Revise the AppVeyor CI configuration to include new job groups for building Flet packages across different platforms and architectures.
  • Introduce a common script for setting up the build environment and managing Python package versions.

Documentation:

  • Add README files for the new Flet CLI, Flet Desktop, and Flet Web packages, providing basic descriptions of their purposes.

Python 3.12

Closes #4007

Packaged Flet app runs on Python 3.12.6 runtime for all platforms.

New packages structure

Closes #3163
Closes #2637

The structure avoids rewriting pip dependencies while installing flet package on various platforms. There was a problem of detecting the correct flet package to install (flet-runtime, flet-embed orflet-pyodide?) if flet was not a direct dependency in user's app.

New Flet packages.

  • flet - required for minimal Flet setup, app entry point for various platforms.
  • flet-core - required for minimal Flet setup, core logic and controls.
  • flet-cli - installed on desktop only, Flet CLI.
  • flet-desktop - installed on desktop only. Contains pre-built Flet "client" app binary for macOS, Windows and Linux.
  • flet-desktop-light - installed on Linux desktop only. Contains a light-weight version of Flet "client" for Linux.
  • flet-web - installed on desktop only. Contains Flet web "client" and FastAPI integration.

Faster re-builds

Closes #4036

Flutter app created by flet build command is not re-created all the time in a temp directory, but cached in build/flutter directory which gives faster re-builds, improves packaging troubleshooting and does not pollute temp directory.

Permissions

  • AndroidManifest.xml for Android.
  • Info.plist for iOS.
  • Info.plist and *.entitlements for macOS.

iOS

No more hard-coded permissions in Info.plist.

Setting iOS permissions:

flet build --info-plist permission_1=True|False|description permission_2=True|False|description ...

For example:

flet build --info-plist NSLocationWhenInUseUsageDescription=This app uses location service when in use.

macOS

No hard-coded entitlements in Release.entitlements.

Setting macOS entitlements:

flet build --macos-entitlements name_1=True|False name_2=True|False ...

Default macOS entitlements:

  • com.apple.security.app-sandbox = False
  • com.apple.security.cs.allow-jit = True
  • com.apple.security.network.client = True
  • com.apple.security.network.server" = True

Android

No hard-coded permissions in AndroidManifest.xml.

Setting Android permissions and features:

flet build --android-permissions permission=True|False ... --android-features feature_name=True|False

For example:

flet build \
    --android-permissions android.permission.READ_EXTERNAL_STORAGE=True \
      android.permission.WRITE_EXTERNAL_STORAGE=True \
    --android-features android.hardware.location.network=False

Default Android permissions:

  • android.permission.INTERNET

Default permissions can be disabled with --android-permissions option and False value, for example:

flet build --android-permissions android.permission.INTERNET=False

Default Android features:

  • android.software.leanback=False (False means it's written in manifest as android:required="false")
  • android.hardware.touchscreen=False

Cross-platform permission groups

There are pre-defined permissions that mapped to Info.plist, *.entitlements and AndroidManifest.xml for respective platforms.

Setting cross-platform permissions:

flet build --permissions permission_1 permission_2 ...

Supported permissions:

  • location
  • camera
  • microphone
  • photo_library

iOS mapping to Info.plist entries

  • location
    • NSLocationWhenInUseUsageDescription = This app uses location service when in use.
    • NSLocationAlwaysAndWhenInUseUsageDescription = This app uses location service.
  • camera
    • NSCameraUsageDescription = This app uses the camera to capture photos and videos.
  • microphone
    • NSMicrophoneUsageDescription = This app uses microphone to record sounds.
  • photo_library
    • NSPhotoLibraryUsageDescription = This app saves photos and videos to the photo library.

macOS mapping to entitlements

  • location
    • com.apple.security.personal-information.location = True
  • camera
    • com.apple.security.device.camera = True
  • microphone
    • com.apple.security.device.audio-input = True
  • photo_library
    • com.apple.security.personal-information.photos-library = True

Android mappings

  • location
    • permissions:
      • android.permission.ACCESS_FINE_LOCATION": True
      • android.permission.ACCESS_COARSE_LOCATION": True
      • android.permission.ACCESS_BACKGROUND_LOCATION": True
    • features:
      • android.hardware.location.network": False
      • android.hardware.location.gps": False
  • camera
    • permissions:
      • android.permission.CAMERA": True
    • features:
      • android.hardware.camera": False
      • android.hardware.camera.any": False
      • android.hardware.camera.front": False
      • android.hardware.camera.external": False
      • android.hardware.camera.autofocus": False
  • microphone
    • permissions:
      • android.permission.RECORD_AUDIO": True
      • android.permission.WRITE_EXTERNAL_STORAGE": True
      • android.permission.READ_EXTERNAL_STORAGE": True
  • photo_library
    • permissions:
      • android.permission.READ_MEDIA_VISUAL_USER_SELECTED": True

"Data" and "Temp" directories for the app

Closes #4114
Closes #2357

Flet developers have been asking where to store application data, such as uploaded files, SQLite databases, etc. that are persistent across application updates.

This release introduce two environment variables that are available in your Flet apps:

  • FLET_APP_DATA - directory for storing application data that is preserved between app updates. That directory is already pre-created.
  • FLET_APP_TEMP - directory for temporary application files, i.e. cache. That directory is already pre-created.

For example, data folder path can be read in your app as:

import os

# it's `None` when running the app in web mode
os.getenv("FLET_APP_DATA")

To make it work in development mode define those variables to suitable locations on your drive, for example:

export FLET_APP_DATA=/path/to/my-app/data
export FLET_APP_TEMP=$TMPDIR
flet run my-app

Deep linking support

Closes #4025

There is a new --deep-linking-url option to configure deep linking for iOS and Android builds. The value must be in the format <sheme>://<host>.

Other features and fixes

Copy link
Contributor

sourcery-ai bot commented Oct 8, 2024

Reviewer's Guide by Sourcery

This pull request implements a significant restructuring of the Flet packages and introduces a new flet build v2 command. The changes include reorganizing the package structure, updating dependencies, modifying build processes, and adjusting how Flet components are imported and used across the project.

Sequence diagram for the new build process in flet_cli

sequenceDiagram
    participant User
    participant CLI as flet_cli
    participant Core as flet_core
    participant Desktop as flet_desktop
    participant Web as flet_web
    User->>CLI: Run flet build
    CLI->>Core: Initialize build environment
    CLI->>Desktop: Prepare desktop build
    CLI->>Web: Prepare web build
    CLI->>CLI: Execute build commands
    CLI->>User: Return build results
Loading

Architecture diagram for the new Flet package structure

graph TD;
    subgraph Flet
        subgraph CLI
            flet_cli --> flet_cli_commands
        end
        subgraph Core
            flet_core --> flet_core_utils
        end
        subgraph Desktop
            flet_desktop
        end
        subgraph Web
            flet_web
        end
    end
    flet_cli --> flet_core
    flet_cli --> flet_desktop
    flet_cli --> flet_web
    flet_desktop --> flet_core
    flet_web --> flet_core
Loading

Class diagram for the updated Command class in flet_cli

classDiagram
    class Command {
        -emojis: dict
        -dart_exe: Optional[str]
        -verbose: bool
        -build_dir: Optional[Path]
        -flutter_dir: Optional[Path]
        -flutter_exe: Optional[str]
        -platforms: dict
        -cross_platform_permissions: dict
        +add_arguments(parser: argparse.ArgumentParser) void
        +handle(options: argparse.Namespace) void
    }
Loading

File-Level Changes

Change Details Files
Restructured Flet packages
  • Created new packages: flet-cli, flet-desktop, and flet-web
  • Removed packages: flet-runtime, flet-embed, and flet-pyodide
  • Updated import statements across the project to reflect new package structure
  • Moved utility functions to more appropriate locations
sdk/python/packages/flet/src/flet/app.py
sdk/python/packages/flet/src/flet/security/__init__.py
sdk/python/packages/flet/src/flet/utils/__init__.py
sdk/python/packages/flet-core/src/flet_core/page.py
sdk/python/packages/flet-cli/src/flet_cli/cli.py
Implemented new flet build v2 command
  • Added support for building desktop and web applications
  • Introduced new command-line arguments for build customization
  • Implemented cross-platform permissions handling
  • Added support for compiling Python files to .pyc
sdk/python/packages/flet-cli/src/flet_cli/commands/build.py
Updated CI/CD pipeline
  • Modified AppVeyor configuration to support new package structure
  • Updated build and test processes for different platforms
  • Added new scripts for patching versions and publishing packages
.appveyor.yml
ci/common.sh
ci/update-flet-wheel-deps.sh
ci/patch_toml_versions.py
ci/patch_toml_package_name.py
Refactored client-side code
  • Updated Flutter dependencies and configurations
  • Modified how Flet components are initialized and used in the client
  • Introduced conditional compilation for different client types (desktop, web)
client/lib/main.dart
client/pubspec.yaml
client/web/index.html
client/web/flutter_bootstrap.js

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @FeodorFitsner - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟡 Documentation: 1 issue found

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

python --version
pip install --upgrade setuptools wheel twine poetry tomlkit virtualenv

function patch_python_package_versions() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Enhance error handling in version patching function

The patch_python_package_versions function is a critical part of the build process. Consider adding error handling and logging to make troubleshooting easier if issues arise during version patching.

function patch_python_package_versions() {
    set -e
    local log_file="/tmp/version_patch.log"
    echo "Starting version patching process..." > "$log_file"
    if ! PYPI_VER="${APPVEYOR_BUILD_VERSION/+/.dev}"; then
        echo "Error: Failed to set PYPI_VER" >> "$log_file"
        return 1
    fi
    echo "PYPI_VER set to: $PYPI_VER" >> "$log_file"
}

Comment on lines 1 to 3
# Flet CLI

Flet CLI.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (documentation): Consider expanding the README with more information about Flet CLI

This README could benefit from additional details about what Flet CLI does, its main features, and basic usage instructions.

# Flet CLI

Flet CLI is a command-line interface tool for Flet, a framework for building interactive multi-platform applications using Python.

## Features
- Create new Flet projects
- Run Flet applications
- Package and deploy Flet apps

## Basic Usage
To create a new Flet project:
flet create myapp

{{flutter_js}}
{{flutter_build_config}}

var loading = document.querySelector('#loading');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Use const or let instead of var. (avoid-using-var)

Explanation`const` is preferred as it ensures you cannot reassign references (which can lead to buggy and confusing code). `let` may be used if you need to reassign references - it's preferred to `var` because it is block- rather than function-scoped.

From the Airbnb JavaScript Style Guide

for member in tar.getmembers():
member_path = os.path.join(path, member.name)
if not is_within_directory(path, member_path):
raise Exception("Attempted Path Traversal in Tar File")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Raise a specific error instead of the general Exception or BaseException (raise-specific-error)

ExplanationIf a piece of code raises a specific exception type rather than the generic [`BaseException`](https://docs.python.org/3/library/exceptions.html#BaseException) or [`Exception`](https://docs.python.org/3/library/exceptions.html#Exception), the calling code can:
  • get more information about what type of error it is
  • define specific exception handling for it

This way, callers of the code can handle the error appropriately.

How can you solve this?

So instead of having code raising Exception or BaseException like

if incorrect_input(value):
    raise Exception("The input is incorrect")

you can have code raising a specific error like

if incorrect_input(value):
    raise ValueError("The input is incorrect")

or

class IncorrectInputError(Exception):
    pass


if incorrect_input(value):
    raise IncorrectInputError("The input is incorrect")



def get_platform():
p = platform.system() if not is_mobile() else ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Swap if/else branches of if expression to remove negation (swap-if-expression)

Suggested change
p = platform.system() if not is_mobile() else ""
p = "" if is_mobile() else platform.system()


ExplanationNegated conditions are more difficult to read than positive ones, so it is best
to avoid them where we can. By swapping the if and else conditions around we
can invert the condition and make it positive.

from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
except ImportError:
raise Exception('Install "cryptography" Python package to use Flet security utils.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Raise a specific error instead of the general Exception or BaseException (raise-specific-error)

ExplanationIf a piece of code raises a specific exception type rather than the generic [`BaseException`](https://docs.python.org/3/library/exceptions.html#BaseException) or [`Exception`](https://docs.python.org/3/library/exceptions.html#Exception), the calling code can:
  • get more information about what type of error it is
  • define specific exception handling for it

This way, callers of the code can handle the error appropriately.

How can you solve this?

So instead of having code raising Exception or BaseException like

if incorrect_input(value):
    raise Exception("The input is incorrect")

you can have code raising a specific error like

if incorrect_input(value):
    raise ValueError("The input is incorrect")

or

class IncorrectInputError(Exception):
    pass


if incorrect_input(value):
    raise IncorrectInputError("The input is incorrect")

Comment on lines +14 to +16
data = f.read(blocksize)
if not data:
break
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): We've found these issues:


def get_arch():
a = platform.machine().lower() if not is_mobile() else ""
if a == "x86_64" or a == "amd64":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Replace multiple comparisons of same variable with in operator (merge-comparisons)

Suggested change
if a == "x86_64" or a == "amd64":
if a in ["x86_64", "amd64"]:

os.remove(pid_file)


def __locate_and_unpack_flet_view(page_url, assets_dir, hidden):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): We've found these issues:


Explanation

The quality score for this function is below the quality threshold of 25%.
This score is a combination of the method length, cognitive complexity and working memory.

How can you solve this?

It might be worth refactoring this function to make it shorter and more readable.

  • Reduce the function length by extracting pieces of functionality out into
    their own functions. This is the most important thing you can do - ideally a
    function should be less than 10 lines.
  • Reduce nesting, perhaps by introducing guard clauses to return early.
  • Ensure that variables are tightly scoped, so that code using related concepts
    sits together within the function rather than being scattered.

@FeodorFitsner FeodorFitsner merged commit 6811d6f into main Oct 9, 2024
2 checks passed
@ndonkoHenri ndonkoHenri linked an issue Oct 10, 2024 that may be closed by this pull request
1 task
@FeodorFitsner FeodorFitsner deleted the flet-new-packages branch October 14, 2024 16:20
@CodeOfHarald
Copy link

Thanks for the new features :)
@FeodorFitsner When can we expect a new release?

@FeodorFitsner
Copy link
Contributor Author

Hopefully next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment