(Ready for Review) Start chat with Folder Import

[wonderwhy-er]

This is proof of concept for folder import
It allows to start a new chat with selecting a folder
First message in the chat will be message similar to one AI returns with file content to create project from

Still needs to be tested + other things to do

• There will be problems with binary files like images, need to ignore them for now
• I have ideas of how to support image and other file uploads in future
• May be we need to ask AI to inspect the project and run commands to show preview?

Video:

2024-11-25.10-23-19.mp4

[mrsimpson]

You may want to include ignore

    try {
      // try to read .gitignore if it exists
      const gitignoreHandle = await sourceHandle.getFileHandle('.gitignore');
      const file = await gitignoreHandle.getFile();
      gitignoreContent = await file.text();
      ig = ignore().add(gitignoreContent);
    } catch (error) {
      // .gitignore doesn't exist, use default ignores
      ig = ignore().add([
        'node_modules',
        '.git',
        'dist',
        'build',
        '.cache',
        '*.log',
        '.DS_Store'
      ]);
    }

[wonderwhy-er]

Good idea, thanks, will add that!

[wonderwhy-er]

Just pushed support for it along with ignoring of binnary files.
I have idea of what to do with binary files but now now. In another PR

[dylanturn]

Is this something we're going to merge soon? I have a couple projects I'd love to import

[wonderwhy-er]

Is this something we're going to merge soon? I have a couple projects I'd love to import

Yeah, avaiting review and testing from others.
Tested it a bit and will be testing more later

[mrsimpson]

What about limits? What happens if – just as an example – you load the projects codebase into it? (Asking for a friend 😬)

Quite sure we're going to see maaaany PRy once this is possible, so maybe get quality assurance in place soon 🔜

[wonderwhy-er]

What about limits? What happens if – just as an example – you load the projects codebase into it? (Asking for a friend 😬)

Quite sure we're going to see maaaany PRy once this is possible, so maybe get quality assurance in place soon 🔜

By limits you mean how much one can load in to context or chat?
If yes then open source bolt is not ready for that yet.
One of next areas of work is context/token management

And yes, playwright/tests too.

And we already have 50 open PRs :D
Need to get to checking and merging some/

[coleam00]

@wonderwhy-er This PR is fantastic - looks awesome and I love how clean your implementation is!

I tested it out myself and it's working perfectly!

I just have a few suggestions, but honestly I would be good with you merging and improving through another PR if you want just because this feature is so sought after:

1. A loading indicator when importing a folder would be sweet, it can take a while especially for larger uploads.
2. Maybe you could polish up the messaging for the user message that says "Import my files" and the Bolt message that say "Sure, I'll help you import these files"? If you referenced the folder name and made it more complete sentences that would be awesome!
3. When you import a project and then ask for it to continue, it doesn't do an npm install or start the server. Maybe we could add into the project somehow when importing a project to tell it that the next time it does anything it needs to run the necessary commands?
4. I tried importing a large project (46,000 files with node_modules, .next, and .git folders being most of them) and it crashed. I see you set up ignoring for these folders, but maybe that isn't enough because those are all still uploaded to the frontend before being filtered? Not totally sure what the best solution would be for these larger projects.

Nice work man!!

[wonderwhy-er]

I agree that we should merge and build on top.
I will add some of the changes you asked as separate PRs
But so far are my thoughts

@coleam00

A loading indicator when importing a folder would be sweet, it can take a while especially for larger uploads.
I trie
I will add that separately but it may not fully help
Bolt.new is not optimized for large projects yet

When you import a project and then ask for it to continue, it doesn't do an npm install or start the server. Maybe we could add into the project somehow when importing a project to tell it that the next time it does anything it needs to run the necessary commands?

Yes, was thinking about it. One challange is that commands are in package.json and they are non standart.
like npm run start or npm run dev are pretty common but what if they are not there?
or if there is not package.json at all?

I will add some handling in separate PR

I tried importing a large project (46,000 files with node_modules, .next, and .git folders being most of them) and it crashed. I see you set up ignoring for these folders, but maybe that isn't enough because those are all still uploaded to the frontend before being filtered? Not totally sure what the best solution would be for these larger projects.

Bolt.new is not optimized for such large projects yet
One big issue before we can even start working with such large proejcts is context size.
Currently whole chat is sent to AI and it will fail with such large projects.
One next thing to do is to tackle that with some kind of form of RAGs
Will take time, that is secret souce of Windsruf and Cursor in comparison to other tools.
I had a thread about here https://thinktank.ottomator.ai/t/some-thoughts-of-where-to-borrow-ideas-and-code-for-developer-features-vs-code-is-open-source/755