Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make yarn-run-dev not move queries.json to build folder #162

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Make yarn-run-dev not move queries.json to build folder #162

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3e68e3d
Make yarn-run-dev not move queries.json to build folder
simonvbrae Sep 19, 2024
3d2aefd
Delete queries.json config linefor build configuration
simonvbrae Sep 19, 2024
039b728
Cleanup
simonvbrae Sep 19, 2024
e612c26
Use correct config for yarn dev-prod script
simonvbrae Sep 27, 2024
fcfb7d8
Test for CI
simonvbrae Oct 11, 2024
5b44f4d
Undo change, can't force CI to run on github
simonvbrae Oct 11, 2024
e97a6e8
Add some debug output
simonvbrae Nov 6, 2024
8a7e47d
remove obsolete addition to buildcontext
simonvbrae Nov 13, 2024
19394a5
remove debug prints
simonvbrae Nov 13, 2024
f3e2b3a
copy queries.json from CWD to fix CI
simonvbrae Nov 22, 2024
04d66b2
Debug path.resolve vs path.join & presence of build folder for CI
simonvbrae Nov 29, 2024
f0d32c6
Change back line that didn't need changing
simonvbrae Nov 29, 2024
167cc95
Create build folder to fix CI
simonvbrae Nov 29, 2024
dfbd305
Copy queries.json from process.cwd()
simonvbrae Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions bin/generate.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ if (args.h || args.help || args._.length > 1) {
const baseURL = args.b || 'https://query.linkeddatafragments.org/';
const webpackConfig = require(args.w ? path.resolve(process.cwd(), args.w) : '../webpack.config.js');

// Remove the location of queries.json which shouldn't be present in the build configuration.
webpackConfig[0].entry=webpackConfig[0].entry.filter(x => !x.endsWith('queries.json'));

// Override the baseURL in the webpack config
webpackConfig.baseURL.replace = baseURL;

Expand All @@ -66,6 +69,7 @@ if (args.h || args.help || args._.length > 1) {
entry.output.path = path.resolve(process.cwd(), destinationPath);
}
}

webpack(webpackConfig, (err, stats) => {
if (err) {
console.error(err.stack || err);
Expand All @@ -82,4 +86,8 @@ if (args.h || args.help || args._.length > 1) {

fs.unlinkSync('.tmp-comunica-engine.js');
});

if (fs.existsSync(path.join(process.cwd(), 'queries.json'))) {
fs.renameSync(path.join(process.cwd(), 'queries.json'), path.join(process.cwd(), `${destinationPath}/queries.json`));
};
})();
2 changes: 1 addition & 1 deletion webpack.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ module.exports = [
path.join(__dirname, './images/sparql.png'),
path.join(__dirname, './favicon.ico'),
path.join(__dirname, './solid-client-id.jsonld'),
path.join(process.cwd(), './queries.json'),
path.join(__dirname, './queries.json'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is correct, as this may break overriding queries.json when using bin/generate.js.
Can you double-check this? (as we don't have unit tests for this)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to still work; using the option -q results in custom queries being made available.

image

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add a unit test for this? And would it be a good start to check if the custom query correctly ends up in queries.json in the build folder?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A test would be great indeed! But let's keep this for a separate PR. I'll merge this one already. Thanks for checking!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry for raising this point again, but the lacking unit test is making me a bit paranoid right now 😅
Could you try invoking bin/generate.js with a custom queries config from a completely different directory to check everything works correctly?

Because the usage of __dirname here would cause the queries.json file to be loaded from the repo directory, and not from a different directory.
But I could be missing something.

Copy link
Author

@simonvbrae simonvbrae Dec 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better safe than sorry!

Calling ./bin/generate -q /home/custom_queries worked correctly.
The reason it works is that queries.json seems to be generated in the directory denoted by __dirname and copied from there to build/.

I thought, what if queries.json gets generated in cwd(), which could be incompatible with using __dirname in some cases. I tested this in two ways:
I ran bin/generate -q /home/custom_queries outside of the repo, this failed because it could not resolve babel-loader, I assume this is normal behaviour?
I also ran .bin/generate from a subdirectory of the repo. This worked correctly.

Do these tests cover everything?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran bin/generate -q /home/custom_queries outside of the repo, this failed because it could not resolve babel-loader, I assume this is normal behaviour?

That should also work, but probably doesn't in this case because it can't find the node_modules.
Can you try globally installing this package (your dev version), and running the bin like that? (yarn link should also be able to do that (at least yarn classic, not sure about the newer ones))
You should then be able to run comunica-web-client-generator from anywhere.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also seemed to work: comunica-web-client-generator -q /home/custom_queries works in any directory and I used a print to make sure it's using my local version correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just re-read your previous comment.
The fact that queries.json is generated in __dirname is a bit strange.
This means that even when running the command elsewhere, queries.json will get generated within the directory of the source code.
I would the source code to remain immutable, and only the cwd to get modified.
Do you think it's feasible to restore the old behaviour of this file getting generated in process.cwd()? This would also simplify debugging on this file.

Copy link
Author

@simonvbrae simonvbrae Dec 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have restored it and tested, everything seems to work correctly, including custom queries folder via -q.

],
output: {
filename: 'scripts/ldf-client-ui.min.js',
Expand Down
Loading