Skip to content

Commit

Permalink
Transfers: implement source selection strategies
Browse files Browse the repository at this point in the history
Try to be more generic in the source filtering and selection code.
Introduce a SourceRankingStrategy class, which has an `apply` method
to actually apply the strategy to a source and decide what to do with
a source. Some strategies are filter-only: i.e. they only decide if
a source has to be ignored or not. Others are also ranking strategies:
they can return an integer which defines the cost of the source
according to this strategy. The order of ranking strategies is
important: it defines the order in which the costs will be compared to
sort the sources.

This change will definitely introduce some computational and memory
overhead, but I hope it will be low. Instead of hard-coded `filter()`
calls which are done once per request, now we have multiple object
traversals and function calls for each source of each request.
  • Loading branch information
Radu Carpa committed Nov 9, 2023
1 parent 4380d02 commit 802221e
Show file tree
Hide file tree
Showing 2 changed files with 296 additions and 151 deletions.
6 changes: 3 additions & 3 deletions lib/rucio/core/request.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@


class RequestSource:
def __init__(self, rse_data, ranking=None, distance=None, file_path=None, scheme=None, url=None):
self.rse = rse_data
def __init__(self, rse: RseData, ranking=None, distance=None, file_path=None, scheme=None, url=None):
self.rse = rse
self.distance = distance if distance is not None else 9999
self.ranking = ranking if ranking is not None else 0
self.file_path = file_path
Expand Down Expand Up @@ -558,7 +558,7 @@ def list_and_mark_transfer_requests_and_source_replicas(
if replica_rse_id is not None:
replica_rse = rse_collection[replica_rse_id]
replica_rse.name = replica_rse_name
source = RequestSource(rse_data=replica_rse, file_path=file_path,
source = RequestSource(rse=replica_rse, file_path=file_path,
ranking=source_ranking, distance=distance, url=source_url)
request.sources.append(source)
if source_rse_id == replica_rse_id:
Expand Down
Loading

0 comments on commit 802221e

Please sign in to comment.