Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FR: Switch to gitoxide backend instead of libgit2 for the backend #2316

Open
nyabinary opened this issue Oct 2, 2023 · 5 comments
Open

FR: Switch to gitoxide backend instead of libgit2 for the backend #2316

nyabinary opened this issue Oct 2, 2023 · 5 comments
Assignees
@yuja
Labels
enhancement New feature or request

Comments

@nyabinary
Copy link

Is your feature request related to a problem? Please describe.
This is obviously a longer term feature request but it would be extremely nice to have a faster implementation of git implemented in jj instead of the current libgit2 backend, the biggest advantages to switching would be performance and the memory safety of the top of my head.

Describe the solution you'd like
Gradual transition to gitoxide as the default backend instead of libgit2

Describe alternatives you've considered
Keep using libgit2

Additional context
gitoxide is dual licensed with MIT and Apache 2.0, making it compatible with this project.
@PhilipMetzger PhilipMetzger added the enhancement New feature or request label Oct 2, 2023
@martinvonz
Copy link
Owner

I suspect it would be easy to switch to gitoxide for reading and writing commits etc. I actually doubt it would have significant impact on performance. It's probably going to be harder to replace libgit2 by gitoxide for some of the other git integration (push/pull, importing/exporting refs).

@nyabinary
Copy link
Author

@martinvonz I mean it claims to be faster so hopefully it is, plus a slow rollout to gitoxide seems like a good idea, replacing libgit2 slowly between versions :3

@thoughtpolice
Copy link
Collaborator

One advantage of Gitoxide, performance or not, is that pure rust code is vastly easier to build and vendor in stranger circumstances; it may even unblock #1997 for instance, which is pretty badly held up on the need for libopenssl-sys to be compiled (which in turn is a dependency of libgit2-rs)

@yuja yuja self-assigned this Oct 30, 2023
yuja added a commit to yuja/jj that referenced this issue Oct 31, 2023
@yuja
git_backend: replace git2::Repository with gix::Repository
328a303 
My gut feeling is that gitoxide aims to be more transparent than libgit2. We'll
need to know more about the underlying Git data model.

Random comments on gix API:

 * gix::Repository provides API similar to git2::Repository, but has less
   "convenient" functions. For example, we need to use .find_object() +
   .try_to/into_<kind>() instead of .find_<kind>().
 * gix::Object, Blob, etc. own raw data as bytes. gix::object and gix::objs
   types provide high-level views on such data.
 * Tree building is pretty low-level compared to git2.
 * gix leverages bstr (i.e. bytes) extensively.

It's probably not difficult to migrate git::import/export_refs(). It might
help eliminate the startup overhead of libssl initialization. The gix-based
GitBackend appears to be a bit faster, but that wouldn't practically matter.

martinvonz#2316
@yuja yuja mentioned this issue Oct 31, 2023
Merged
4 tasks
yuja added a commit to yuja/jj that referenced this issue Oct 31, 2023
@yuja
git_backend: replace git2::Repository with gix::Repository
e6dd022 
My gut feeling is that gitoxide aims to be more transparent than libgit2. We'll
need to know more about the underlying Git data model.

Random comments on gix API:

 * gix::Repository provides API similar to git2::Repository, but has less
   "convenient" functions. For example, we need to use .find_object() +
   .try_to/into_<kind>() instead of .find_<kind>().
 * gix::Object, Blob, etc. own raw data as bytes. gix::object and gix::objs
   types provide high-level views on such data.
 * Tree building is pretty low-level compared to git2.
 * gix leverages bstr (i.e. bytes) extensively.

It's probably not difficult to migrate git::import/export_refs(). It might
help eliminate the startup overhead of libssl initialization. The gix-based
GitBackend appears to be a bit faster, but that wouldn't practically matter.

martinvonz#2316
yuja added a commit to yuja/jj that referenced this issue Nov 2, 2023
@yuja
git_backend: replace git2::Repository with gix::Repository
fdf5915 
My gut feeling is that gitoxide aims to be more transparent than libgit2. We'll
need to know more about the underlying Git data model.

Random comments on gix API:

 * gix::Repository provides API similar to git2::Repository, but has less
   "convenient" functions. For example, we need to use .find_object() +
   .try_to/into_<kind>() instead of .find_<kind>().
 * gix::Object, Blob, etc. own raw data as bytes. gix::object and gix::objs
   types provide high-level views on such data.
 * Tree building is pretty low-level compared to git2.
 * gix leverages bstr (i.e. bytes) extensively.

It's probably not difficult to migrate git::import/export_refs(). It might
help eliminate the startup overhead of libssl initialization. The gix-based
GitBackend appears to be a bit faster, but that wouldn't practically matter.

martinvonz#2316
yuja added a commit that referenced this issue Nov 2, 2023
@yuja
git_backend: replace git2::Repository with gix::Repository
c88e69a 
My gut feeling is that gitoxide aims to be more transparent than libgit2. We'll
need to know more about the underlying Git data model.

Random comments on gix API:

 * gix::Repository provides API similar to git2::Repository, but has less
   "convenient" functions. For example, we need to use .find_object() +
   .try_to/into_<kind>() instead of .find_<kind>().
 * gix::Object, Blob, etc. own raw data as bytes. gix::object and gix::objs
   types provide high-level views on such data.
 * Tree building is pretty low-level compared to git2.
 * gix leverages bstr (i.e. bytes) extensively.

It's probably not difficult to migrate git::import/export_refs(). It might
help eliminate the startup overhead of libssl initialization. The gix-based
GitBackend appears to be a bit faster, but that wouldn't practically matter.

#2316
@nyabinary
Copy link
Author

What's the status on this being possible?

@martinvonz
Copy link
Owner

It's partially done. Most local operations use Gitoxide, but we haven't started using it for push/pull yet mostly because Gitoxide doesn't support push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yuja yuja

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@thoughtpolice @yuja @martinvonz @PhilipMetzger @nyabinary