Skip to content

Commit

Permalink
Fix performance in get_imports regexp (#34298)
Browse files Browse the repository at this point in the history
* fix: Fix performance in get_imports regexp

* Minimize get_imports content regexp
  • Loading branch information
AlekseyLobanov authored Oct 29, 2024
1 parent 4e2e880 commit f55595b
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/transformers/dynamic_module_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,8 @@ def get_imports(filename: Union[str, os.PathLike]) -> List[str]:
content = f.read()

# filter out try/except block so in custom code we can have try/except imports
content = re.sub(r"\s*try\s*:\s*.*?\s*except\s*.*?:", "", content, flags=re.MULTILINE | re.DOTALL)
content = re.sub(r"\s*try\s*:.*?except.*?:", "", content, flags=re.DOTALL)

# filter out imports under is_flash_attn_2_available block for avoid import issues in cpu only environment
content = re.sub(
r"if is_flash_attn[a-zA-Z0-9_]+available\(\):\s*(from flash_attn\s*.*\s*)+", "", content, flags=re.MULTILINE
Expand Down

0 comments on commit f55595b

Please sign in to comment.