File Set Request (Files With Links to Other Files In the Same Directory)

[Mikenux]

Note: This proposal currently does not apply to files with links to files from different directories. It can be opened for those if there is no problem doing so (e.g. for security reasons).

Goal

The goal is to allow apps to request access to a set of files (i.e. multiple files at once) after opening a file requiring it. The opened file and requested files are all in the same directory.

Requirements

Portal

• See the "Functioning", "Files Added After Fileset Request", "Open Fileset Method", "Delay Before Responding", "Portal UI With Portal Code", and "Waiting Window" sections below.

Fileset Request

• A fileset request consists of a list of file paths.
• The files are located in the same directory.
• The file list contains only file names; the use of wildcards is not allowed.
• Perhaps for better performance, this file list should be located at the beginning of the files containing this list.

Example file list:

file1.ext
file2.ext
file3.ext
folder-1/file4.ext
folder-1/file5.ext
folder-2/file6.ext
folder-2/file7.ext
folder-2/folder-3/file8.ext

Apps

• Use of the file chooser portal.
• Support for requesting a set of files (i.e. requesting multiple files at once).
• Have a reference file in which are written the file types for which a request for a set of files will be made. All full extensions and suffixes are written. Example: file-set: cue, m3u, edl, _project.

Functioning

Note: The following text is for understanding purposes only. Things can be done differently if wanted; only the end results are important.

Checks and their values:

• (1) The type of the file opened by the user is defined in the app reference file in "file-set". Values: "no", "yes".
• (2) The app requests a set of files within a defined time limit (to be defined) which starts when the file selected by the user has been returned to the app. Values: "no", "yes".
• (3a) The request contains only paths to files with precise names, i.e. only names of files in their directory; any wildcard is not allowed. Values: "no" or "yes".
• (3b) The request concerns files located in the same directory as the file opened by the user - their existence is not checked. Values: "no" or "yes".
• (4a) There are missing requested files. Values: "none" if no file is missing; "some" if some are missing, and "all" if all are missing.
• (4b) Each requested file has a "locked" access permission in the permission store. Global values: "none" if no file has a "locked" permission, "some" if some files have a "locked" permission, but others do not, "all" if all files have a "locked" access.

To begin, the user opens a file containing links to other files (which are in the same directory) with the file manager, the file chooser, drag and drop, or the open-fileset method (see below). Then the portal performs check (1). If this check has the value "no", the portal returns the file opened by the user to the app. However, if it has the value "yes", the portal returns the file to the app and performs check (2).

The app parses the file opened by the user and makes its request for a set of files within the time limit imposed by check (2). If check (2) has the value "no", meaning the app has not made its request within the specified time limit, the portal denies this request. If check (2) has the value "yes", which means that the app request was made within the time limit, the portal performs check (3a).

With check (3a) having the value "no", the portal denies this fileset request. If this check has the value "yes", then the portal does check (3b).

If check (3b) has the value "no", the portal denies the request. However, if this check has the value "yes", then the portal does checks (4a) and (4b) at the same time.

Finally, the portal has one of the following behaviors depending on the values of checks (4a) and (4b), classified according to the values of check (4a).

When (4a) is "none", and:

• (4b) is "none": The portal displays a dialog box displaying all files requested for the file opened by the user. This dialog box tells the user to check if there are any files that should not have been requested, which indicates something suspicious. Then the user decides whether or not to grant access to the requested files. If the user has granted access, the portal returns all requested files to the app and stores their access permissions by attaching them to that of the file opened by the user. If the user has denied access, the portal does not return the requested files to the app.
• (4b) is "some": The portal displays a dialog box showing the files requested for the file opened by the user that do not have a "locked" permission. Files that have a "locked" permission may also be displayed, in order to help the user better remember whether these new files have actually been added to the fileset. This dialog box tells the user to check if there are any files that should not have been requested, which indicates something suspicious. Then the user decides whether or not to grant access to the requested files. If the user has granted access, the portal returns all requested files to the app and stores their access permissions by attaching them to that of the file opened by the user. If the user has denied access, the portal does not return the requested files to the app.
• (4b) is "all": The portal returns all requested files to the app and stores their access permissions by attaching them to that of the file opened by the user.

When (4a) is "some", and:

• (4b) is "none": The portal displays a dialog box displaying all requested files with missing ones in a separate section for the file opened by the user. This dialog box tells the user to check if there is anything suspicious: are there any files that should not have been requested? do the missing files exist, have they been moved, or have they been deleted? Then the user decides whether or not to grant access to the requested files. If the user has granted access, the portal returns all requested files to the app and stores their access permissions by attaching them to that of the file opened by the user. If the user has denied access, the portal does not return the requested files to the app.
• (4b) is "some": The portal displays a dialog box showing the files requested for the file opened by the user that do not have a "locked" permission. Missing files are distinguished from existing files. Files that have a "locked" permission can also be displayed with the same distinction for missing files, in order to help the user better remember whether these new files have actually been added to the fileset. This dialog box tells the user to check if there are any files that should not have been requested, which indicates something suspicious. Then the user decides whether or not to grant access to the requested files. If the user has granted access, the portal returns all requested files to the app and stores their access permissions by attaching them to that of the file opened by the user. If the user has denied access, the portal does not return the requested files to the app.
• (4b) is "all": The portal displays a dialog box indicating which files are missing and indicating that they have been moved or deleted (because they have already been accessed by the app). The user validates having been informed of this. Then, the portal returns all existing requested files to the app and stores their access permissions by attaching them to that of the file opened by the user.

When (4a) has "all", and:

• (4b) is "none": The portal displays a dialog box showing all missing requested files for the file opened by the user. This dialog box indicates that no files can be returned because they are missing and allows to check if these files already existed (which is suspicious), have been moved, or have been deleted. The user then validates that they has been informed that the files are not accessible.
• (4b) is "some": The portal displays a dialog box stating that all requested files are missing and some of them were never accessed during a previous opening of the file opened by the user, which is suspicious. The dialog box shows all missing requested files, with those not having a "locked" access permission shown in a separate section which is the first to be displayed. The user then validates that they has been informed of the access denial.
• (4b) is "all": The portal displays a dialog box displaying all missing requested files for the file selected by the user. This dialog box indicates that no files can be returned because they have been moved or deleted (this can be indicated because all files have already been accessed before).

When the Request Is Suspicious Or Automatically Denied

In case there is something suspicious or the app request was automatically denied, the dialog box can also suggest actions to take, such as checking if the file contains links to the requested files (if it is readable by a text editor), checking with another file in case the open one has been modified with another app or by another process, uninstalling the app, and any other relevant actions that can be taken.

Approved Request and Files Added After Request

When a valid fileset request is approved by the user, any file from this request and added after this request has its access permission marked as "locked" by the portal.

Note that files with "locked" permission can be opened independently of any fileset request.

Open Fileset Method

The open-fileset method allows to perform a fileset request for any file the app has access to.

Delay Before Responding

A delay during which the user cannot deny or approve a request is written into the portal code. This delay is the minimum time required for a user to read a filename multiplied by the number of files requested, plus the minimum time required to read a folder name multiplied by the total number of folders involved in the request (e.g., if there are the folders "folders -1", "folder-2" and "folder-1/folder-3", there are 3 folders). This delay is limited to valid requests for which a dialog box is displayed to approve or deny them.

Portal User Interface With Portal Code

This involves having a transaction between the portal UI and the portal code to present the files to the user in the portal UI. This allows (in part) to create the same type of user experience across different desktop environments.

Here are a few suggestions, which can be combined:

• Show a given number of files per setup page. Maybe 10-15 files per page. In addition to the delay, this can improve user review of files by possibly reducing their feeling of having too many things to check at once (i.e. showing all files on the same interface page).
• Show files by folder (i.e. organize files by folders they belong to). One folder per page, or more depending on the number of files.
• Show files deemed sensitive such as dot files, dot directories and scripts per page. One file/folder type deemed sensitive per setup page. This is incompatible with the second suggestion, resulting in a loss of context for files in the same folder. This has the advantage that anything sensitive is identified by the portal.
• Show missing files on separate setup page(s). This also has the disadvantage that these files cannot be displayed in relation to other files of the same folder. However, it is one of the key points of the portal.

The functioning is as follows when using multiple setup pages:

1. The portal code returns the relevant files for the first setup page to the portal UI.
2. When the user clicks, for example, on a "Next" button in the portal UI, the portal returns a "next" signal to the portal code.
3. The portal code returns the relevant files for the second page to the portal UI, and so on.
4. On the last setup page, the portal UI displays buttons to deny or approve the fileset request and returns the user's chosen response to the portal code.

Waiting Window

Each time a request is made, including when opening files without user intervention, use of the app UI is blocked due to request processing. For this, a small window displaying a short message, like "Processing fileset request..." with the name of the file that "triggered" the request, might be displayed before showing the result of the request. The display time of this window must be sufficient, without being too long, for the user to perceive the message. The transition between the wait window and the request result should be smooth. The blocking is stopped when the files are returned to the app.

I recommend doing this because it would let the user know something is happening every time.

Additionally, one option for apps might be added:

• wait-more. This option allows an app to indicate to the portal that the blocking of the app's UI should last longer, because it may process multiple fileset requests one after the other. A duration should be defined for this.

Recommendations for App Developers

Multiple Requests

For each request, the above functioning applies. As such, it is recommended, if possible, to trigger a request using the open-fileset method when the user opens a file (e.g. double-click a file in the app UI). Chaining requests (i.e. to perform requests directly one after the other) should be avoided.

Links To Files Not Requiring Direct Access

This is, for example, the case for external attachments to PDF files. In such cases, it might be better to use the OpenURI portal ("Open with") rather than using this one.

If this is not the case, it is necessary for the OpenURI portal to support the opening of files that the app does not have access to and which are within the same directory. This opening should always be explicitly approved or denied by the user. Note that it is better to have this at the same time as the Fileset Request portal.

Notes

This could be cumbersome for the user:

• To ask it to examine the files requested for each file it opens, especially if they has many files with links to other files.
• To spot if there is anything suspicious in the case of files requiring the opening of a large number of files.

Questions

• Is it possible for toolkits and/or for parsing libraries to implement file set request? If so, is it possible to do it in such a way that app developers can adapt their apps with minimal changes and it is compatible with cross-platform apps?
• Is it really useful to change file permissions to "locked" instead of just keeping file permissions as is?
• Is there anything else that should make a fileset request invalid?
• Are there other types of files/folders that are considered sensitive and can be easily identified by the portal code and/or portal user interface?
• Is a "wait-until-ready" option to the wait window necessary? This option would block use of the app's UI until the app is ready to display files in its UI. When it is ready, it sends a signal to the portal to stop the blocking.

[Mikenux]

The proposal has been updated:

• Small wording changes.
• Mention of checks only once.
• When the value of check (4) is "all", the value of check (1) is any.
• Added removal of access to requested files when the app is no longer running.

The changes can be known using the “edited” button.

[Mikenux]

Update 2:

• Clarification.
• Added "A Simpler Portal" (do not implement the "confirm" access permission).

[Mikenux]

Update 3:

• Major update! Please read again.

[Mikenux]

Update 4:

• New section: Fileset Request.
• New check (3a) in Functioning section.
• New section: Open Fileset Method.
• New section: Delay Before Responding.
• New section: Portal User Interface With Portal Code.
• New section: Waiting Window.
• New section: Recommendations for App Developers.
• Two new Questions.

[Mikenux]

Update 5:

• New section: Files Added After Fileset Request.
• Waiting Window section: Clarifications and one option added.
• Questions: Added a question regarding a second option for the waiting window.

[Mikenux]

Update 6:

• The proposal can optionally be opened for a file set where the files are located in different locations.
• Opening file sets where some files have not been accessed before is allowed. Blocking this did not make sense because a file set where the files are all in the same location is usually portable and therefore editable on different computers.
• Clarification.

These changes (made the same day as this comment) can be viewed using the "edited" button.

[adrianinsaval]

I'm not familiar with the process for proposals here, is this something that is being worked on? What needs to happen so it gets worked on?

Beyond that I have some questions, I'm from the perspective of a freecad user/dev trying to understand if links can eventually work with the desktop portal. Currently our flatpak app just has host permissions to wrokaround this.

An app can only make one request for a set of files after the user opens a file via the file chooser, file manager, or drag and drop. Once a request has been processed, whether it has failed, been denied, or access has been granted, any further requests from the app will be automatically denied until the user opens another file via the file chooser, file manager or drag and drop.

what if a linked file also links to other files? This is common in freecad projects. The app would need to read the linked file before knowing it links to other files so it will need to make more than one request to get all the linked files it needs, I don't think secondary requests should be automatically refused, in any case the user could be prompted (and given the option to accept all requests)

(2) The app requests a set of files within a defined time limit (to be defined) which starts when the file selected by the user has been returned to the app. Values: "no", "yes".

If a time limit will be imposed I hope it's a relatively long one or that there is still the possibility of requesting access with a prompt for the user if the time has passed. CAD files can get pretty big and therefore slow to open in big projects, although I'm not sure if links are read and requested early. Considering the case of linked files linking others I mentioned before I think the timer should be reset after the first level of linked files is accessed.

Would this also allow access to files in sub-directories in the same location? This is also common in FreeCAD projects and if I'm being honest any arbitrary relative path from the file is a common thing, can we think about allowing arbitrary relative paths? We can think about prompting the user whenever the app request files in a different directory.

This could be cumbersome for the user:

To ask it to examine the files requested for each file it opens, especially if they has many files with links to other files.
To spot if there is anything suspicious in the case of files requiring the opening of a large number of files.

we could be smart about this, an app requesting a bunch of files of the same extension as the opened file is not so suspicious, the files could be grouped by extensions and the interface could have more highlight to specially suspicious requests like scripts, dot files and directories or anything deemed sensitive

[Mikenux]

Thanks for you interest!

is this something that is being worked on? What needs to happen so it gets worked on?

Unfortunately, I'm not a developer and I don't know if anyone is working on it, or what needs to happen for this.

what if a linked file also links to other files?

The point is that the delay should not be too long, so that the user knows that this is a file to open other files. This meaning is difficult to convey if it is too long or if the user has no indication. This is done to not give users the impression that there is a problem (since with the portal we go from not asking when opening the "project" file to asking when opening it).

Perhaps, given in the proposal that the portal only waits if the file is a "project" file, we can simply display a modal window indicating that the app is processing the file before displaying the requested files. And so on for the following files. Note this does not remove the idea of the delay. What do you think of this idea?

(and given the option to accept all requests)

I'm not in favor of having the option to accept all requests, as that may defeat the protective purpose of the portal (i.e. asking when there might be something suspicious ). Are there so many requests to be made? Do all requests have to be made one after the other? Given the idea in my answer above, maybe I can add a method to make the app perform a request for a given file; for example, the app performs the request when the user clicks on the "project" file.

CAD files can get pretty big and therefore slow to open in big projects, although I'm not sure if links are read and requested early.

Ideally, a request should request all files as quickly as possible. This might require changes to the code that parses a file. The developer who wants to develop this proposal should take into account developers who do not want to make big changes to their apps, more so for cross-platform apps. The time limit to define in the proposal depends on it.

Would this also allow access to files in sub-directories in the same location?

Yes.

We can think about prompting the user whenever the app request files in a different directory.

If this directory is in the same location as the "project" file, no problem. If it's one or more directories from different locations than the "project" file (like a video project), no.

we could be smart about this, an app requesting a bunch of files of the same extension as the opened file is not so suspicious, the files could be grouped by extensions and the interface could have more highlight to specially suspicious requests like scripts, dot files and directories or anything deemed sensitive

Other files may be sensitive to a user because we don't know how they organizes their files. For your suggestion of simplification, why not, perhaps we could simplify after a given number of files requested.

[Mikenux]

Regarding dot files, is there a valid reason for an app to request them when requesting non-dot files at the same time?

[ptomato]

Regarding dot files, is there a valid reason for an app to request them when requesting non-dot files at the same time?

Yes. An Inform XYZ.materials folder may have a .gitignore file inside it, which is autogenerated by the Inform compiler.

[Mikenux]

@ptomato: Did you mention Inform to tell me that there is the case of development, or because Inform is concerned by this proposal? (just to be sure)

[ptomato]

Yes, I'm following this proposal because I'm assuming it's what I'd need in order to allow Inform to access the sibling .materials folder.

[Mikenux]

So Inform isn't a case of naming matching (#1290) then?

[ptomato]

Possibly? I'm just generally trying to follow along anywhere that might be relevant.

[Mikenux]

Ok