[pull] master from JuliaLang:master #1814
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
This PR contains some refactoring of common functions that were moved to `gc-common.c` and should be shared between MMTk and Julia's stock GC.
Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: 27c1b1ee5 New commit: 799dc2d54 Julia version: 1.12.0-DEV Pkg version: 1.12.0 Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@27c1b1e...799dc2d ``` $ git log --oneline 27c1b1ee5..799dc2d54 799dc2d54 REPLExt: use Base.isaccessibledir rather than isdir in completions (#4053) 3fde94ee9 REPLExt: run repl hint generation for modeswitch chars when not switching (#4054) ``` Co-authored-by: Dilum Aluthge <[email protected]>
…o! (#56237) This might help llvm understand whats going on. Also enzyme really wants this to look like this to trace through it better. --------- Co-authored-by: Jameson Nash <[email protected]>
This fixes a bug introduced in #55941. We may also take this opportunity to limit the scope of the `@inbounds` annotations, and also use `axes` to compute the bounds instead of hard-coding them. The real "fix" here is on line 767, where `l in 1:nA` should have been `l in 1:mB`. Using `axes` avoids such errors, and makes the operation safer as well.
) Currently, `::Diagonal * ::AbstractMatrix` calls the method `LinearAlgebra.__muldiag!` in general that scales the rows, and similarly for the diagonal on the right. The implementation of `__muldiag` was duplicating the logic in `LinearAlgebra.modify!` and the methods for `MulAddMul`. This PR replaces the various branches with calls to `modify!` instead. I've also extracted the multiplication logic into its own function `__muldiag_nonzeroalpha!` so that this may be specialized for matrix types, such as triangular ones. Secondly, `::Diagonal * ::UpperTriangular` (and similarly, other triangular matrices) was specialized to forward the multiplication to the parent of the triangular. For strided matrices, however, it makes more sense to use the structure and scale only the filled half of the matrix. Firstly, this improves performance, and secondly, this avoids errors in case the parent isn't fully initialized corresponding to the structural zero elements. Performance improvement: ```julia julia> D = Diagonal(1:400); julia> U = UpperTriangular(zeros(size(D))); julia> @Btime $D * $U; 314.944 μs (3 allocations: 1.22 MiB) # v"1.12.0-DEV.1288" 195.960 μs (3 allocations: 1.22 MiB) # This PR ``` Fix: ```julia julia> M = Matrix{BigFloat}(undef, 2, 2); julia> M[1,1] = M[2,2] = M[1,2] = 3; julia> U = UpperTriangular(M) 2×2 UpperTriangular{BigFloat, Matrix{BigFloat}}: 3.0 3.0 ⋅ 3.0 julia> D = Diagonal(1:2); julia> U * D # works after this PR 2×2 UpperTriangular{BigFloat, Matrix{BigFloat}}: 3.0 6.0 ⋅ 6.0 ```
Now that I've had a few months to recover from the slog of adding `BindingPartition`, it's time to renew my quest to finish #54654. This adds the basic infrastructure for having multiple partitions, including making the lookup respect the `world` argument - on-demand allocation of missing partitions, `Base.delete_binding` and the `@world` macro. Not included is any inference or invalidation support, or any support for the runtime to create partitions itself (only `Base.delete_binding` does that for now), which will come in subsequent PRs.
Since `zero(::Union{Missing,T})` calls `zero(T)` internally, we may use
the same logic for `LinearAlgebra.haszero`. This helps with certain
structured matrix operations:
```julia
julia> M = Matrix{Union{Int,Missing}}(missing,2,2)
2×2 Matrix{Union{Missing, Int64}}:
missing missing
missing missing
julia> triu(M)
2×2 Matrix{Union{Missing, Int64}}:
missing missing
0 missing
```
whereas previously, this would have been
```julia
julia> triu(M)
2×2 Matrix{Union{Missing, Int64}}:
missing missing
missing missing
```
This comments out an assert thats currently faulty and also marks apply_iterate as safe when we can special case it to not dispatch
After this, ```julia julia> zeros(-1) ERROR: ArgumentError: invalid GenericMemory size: the number of elements is either negative or too large for system address width [...] ``` The error message is updated to warn about possible negative sizes when creating arrays. Fixes #55446
…ag`" (#56267) Reverts #55984 This PR was buggy, but the test failures as seen in https://buildkite.com/julialang/julia-master/builds/41300#0192adab-9d07-4900-8592-2d46aff26905 were not caught in the CI run for the PR as the tests are run randomly. Let's revert this for now.
Assuming that block matrices contain tiled elements (i.e. all elements
along a row have the same number of rows and similarly, all elements
along a column have the same number of columns), we may generalize
`diagzero` to accept arbitrary matrices. We may therefore use only the
diagonal elements to generate the structural zeros. This was being
assumed anyway in the individual methods.
We also now use `diagzero` in indexing triangular matrices, so the
following would work correctly:
```julia
julia> M = reshape([ones(2,2), fill(2,4,2), fill(3,2,3), fill(4,4,3)],2,2)
2×2 Matrix{Matrix{Float64}}:
[1.0 1.0; 1.0 1.0] … [3.0 3.0 3.0; 3.0 3.0 3.0]
[2.0 2.0; 2.0 2.0; 2.0 2.0; 2.0 2.0] [4.0 4.0 4.0; 4.0 4.0 4.0; 4.0 4.0 4.0; 4.0 4.0 4.0]
julia> U = UpperTriangular(M)
2×2 UpperTriangular{Matrix{Float64}, Matrix{Matrix{Float64}}}:
[1.0 1.0; 1.0 1.0] … [3.0 3.0 3.0; 3.0 3.0 3.0]
⋅ [4.0 4.0 4.0; 4.0 4.0 4.0; 4.0 4.0 4.0; 4.0 4.0 4.0]
julia> U[2,1]
4×2 Matrix{Float64}:
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
julia> U[2,1] == zero(M[2,1])
true
```
This also changes
```julia
julia> M = Matrix{Union{Int,Missing}}(missing,4,4)
4×4 Matrix{Union{Missing, Int64}}:
missing missing missing missing
missing missing missing missing
missing missing missing missing
missing missing missing missing
julia> U = UpperTriangular(M)
4×4 UpperTriangular{Union{Missing, Int64}, Matrix{Union{Missing, Int64}}}:
missing missing missing missing
⋅ missing missing missing
⋅ ⋅ missing missing
⋅ ⋅ ⋅ missing
julia> U[3,1] # v"1.12.0-DEV.1373"
missing
```
to
```julia
julia> U[3,1] # this PR
0
```
This call resolution code was deleted in #56179 (rightfully so), but it should be a no-op until we implement this in inference.
Backport already on #56228
All ecosystem consumers have switched to the string-based API.
…ckages are in the sysimage (#52841) (#56234) This reverts commit 08d229f. There are some bugs now where extensions do not load when their package has been put into the sysimage. #52841 was made because it was common to get cycles otherwise but with #55589 that should be much less of a problem. Subsumes #54750.
this should be safer for the type deduction and fixes a build error for macos on Yggdrasil (JuliaPackaging/Yggdrasil#9660): ``` src/jitlayers.cpp:665:54: error: no viable constructor or deduction guide for deduction of template arguments of 'tuple' 665 | incompletemodules.insert(std::pair(codeinst, std::tuple(std::move(params), waiting))); ``` The Yggdrasil environment is a bit special with a rather new clang (version 17) but an old macos sdk and I don't know exactly in which circumstances this triggers. But I think `std::make_tuple` should be more reliable when the tuple types are not specified. cc: @fingolfin
…ractiveUtils (#56276) This way all packages can be timed including InteractiveUtils and its deps (Base64, JuliaSyntaxHighlighting, Markdown, StyledStrings). With this PR ``` % ./julia --start=no -e "@time Base.@time_imports using REPL" 41.8 ms StyledStrings ┌ 0.1 ms JuliaSyntaxHighlighting.__init__() 14.2 ms JuliaSyntaxHighlighting 1.0 ms Base64 ┌ 0.0 ms Markdown.__init__() 9.6 ms Markdown 2.2 ms InteractiveUtils 0.3 ms Unicode ┌ 0.0 ms REPL.REPLCompletions.__init__() ├ 0.0 ms REPL.__init__() 95.7 ms REPL 0.225907 seconds (290.95 k allocations: 16.761 MiB) ``` Otherwise ``` % ./julia --start=no -e "using InteractiveUtils; @time @time_imports using REPL" 0.5 ms Unicode ┌ 0.0 ms REPL.REPLCompletions.__init__() ├ 0.1 ms REPL.__init__() 107.5 ms REPL 0.127016 seconds (164.18 k allocations: 9.199 MiB) ``` Also the `@trace_compile` and `@trace_dispatch` macros for the same reason.
This change duplicates `finally` blocks in lowered IR, so that they can
have a static nesting depth in the `try-catch` hierarchy.
Previously, `finally` control-flow looked like this:
```
error non-error
\ /
\ /
|
finally block
|
/ \
/ \
error non-error
```
This kind of flow is a problem, because in a couple places the compiler
assumes that it can actually "color" the CFG such that there is a static
nesting depth at each BasicBlock (i.e. each BasicBlock can be labeled w/
a unique enclosing `try` / `catch` scope). The above `finally` pattern
violates that assumption.
In an upcoming PR, I want to extend the lifetimes of our Event Handlers
(`jl_handler_t`) until the end of a `catch` block (rather than the
start) which noticeably breaks `llvm-lower-handlers.cpp`. (@Keno was
very clear about this assumption in the comments for that pass.)
Behaviorally this was _mostly_ benign, except for some mis-handling of
an erroring entry that turns into a non-erroring exit. That could be fixed
by banning `break` and `return` within `finally` blocks or making the
lowering more complicated, but this PR instead splits the `finally` block
into an erroring and non-erroring path so that we can attach the `catch`
handler appropriately.
…56275) Update: Just adds the comment in color terminal mode too --- I didn't think adding the `# recompile` text to the end in the repl was a good idea as it increases likelihood of text wrapping. And the color should be sufficient for local review, but when people copy from a color terminal we lose the recompile info. So this just adds a zero-length change indicator, for people to look out for.
1. Allow fully qualified module names: `@world(Foo.Bar.baz, 1)` 2. Correct the printing order of the world macro and module qualifier 3. Add pretty printing for Binding(Partition). Example of the test: ``` julia> GlobalRef(Rebinding, :Foo).binding Binding Main.Rebinding.Foo 27497:∞ - undefined binding - guard entry 0:27496 - constant binding to @world(Main.Rebinding.Foo, 0:27496) ``` --------- Co-authored-by: Shuhei Kadowaki <[email protected]>
I want to move out LinearAlgebra into its own repository but I still want to be able to run its tests in parallel. The easiest would be to be able to use `Base.runtests` but right now it hard codes the path to the stdlib folder. Instead, use the loading mechanism to look up where the stdlib of the active project actively resides.
#56409 broke PackageCompiler (or other use cases where you want to compile a new core compiler from a release build) since it hardcoded the relative path `../usr/` from Base to the `shared` directory but this is not true in releases where it is at `..`.
- incorrect edge types were being added from inlining: there is minimal dispatch info available, so best not to add that (which was already added earlier) as it results in failures to validate later - MethodTable/sig order in edges could confuse the iterator: always put the type before the edge now as that is more consistent - edges wasn't converted to a SimpleVector, so they might get ignored later from being in the wrong format - edges were not populated for optimize=false, which made debugging them more inconvenient Fixes #56577
Previously our precompilation code was causing anything with package A
as a dependency to wait on all of A's extensions and weakdeps to finish
before starting to pre-compile, even if it can't actually load those
weakdeps (or the extensions themselves)
This would lead to a pre-compile ordering like:
```
A B
\ / \
Ext AB \
| /
C /
\ /
D
```
Here `C` cannot pre-compile in parallel with `Ext {A,B}` and `B`,
because it has to wait for `Ext {A,B}` to finish pre-compiling. That
happens even though `C` has no way to load either of these.
This change updates the pre-compile ordering to be more parallel,
reflecting the true place where `Ext {A,B}` can be loaded:
```
A B
/ \ / \
C Ext AB |
\ | /
\-- D --/
```
which allows `C` to compile in parallel with `B` and `Ext{A,B}`
This PR introduces a new, toplevel-only, syntax form `:worldinc` that semantically represents the effect of raising the current task's world age to the latest world for the remainder of the current toplevel evaluation (that context being an entry to `eval` or a module expression). For detailed motivation on why this is desirable, see #55145, which I won't repeat here, but the gist is that we never really defined when world-age increments and worse are inconsistent about it. This is something we need to figure out now, because the bindings partition work will make world age even more observable via bindings. Having created a mechanism for world age increments, the big question is one of policy, i.e. when should these world age increments be inserted. Several reasonable options exist: 1. After world-age affecting syntax constructs (as proprosed in #55145) 2. Option 1 + some reasonable additional cases that people rely on 3. Before any top level `call` expression 4. Before any expression at toplevel whatsover As an example, case, consider `a == a` at toplevel. Depending on the semantics that could either be the same as in local scope, or each of the four world age dependent lookups (three binding lookups, one method lookup) could (potentially) occur in a different world age. The general tradeoff here is between the risk of exposing the user to confusing world age errors and our ability to optimize top-level code (in general, any `:worldinc` statement will require us to fully pessimize or recompile all following code). This PR basically implements option 2 with the following semantics: 1. The interpreter explicit raises the world age only at `:worldinc` exprs or after `:module` exprs. 2. The frontend inserts `:worldinc` after all struct definitions, method definitions, `using` and `import. 3. The `@eval` macro inserts a worldinc following the call to `eval` if at toplevel 4. A literal (syntactic) call to `include` gains an implicit `worldinc`. Of these the fourth is probably the most questionable, but is necessary to make this non-breaking for most code patterns. Perhaps it would have been better to make `include` a macro from the beginning (esp because it already has semantics that look a little like reaching into the calling module), but that ship has sailed. Unfortunately, I don't see any good intermediate options between this PR and option #3 above. I think option #3 is closest to what we have right now, but if we were to choose it and actually fix the soundness issues, I expect that we would be destroying all performance of global-scope code. For this reason, I would like to try to make the version in this PR work, even if the semantics are a little ugly. The biggest pattern that this PR does not catch is: ``` eval(:(f() = 1)) f() ``` We could apply the same `include` special case to eval, but given the existence of `@eval` which allows addressing this at the macro level, I decided not to. We can decide which way we want to go on this based on what the package ecosystem looks like.
This does two things: 1. Forward `copytrito!` for triangular matrices to the parent in case the specified `uplo` corresponds to the stored part. This works because these matrices share their elements with the parents for the stored part. 2. Make `copytrito!` only copy the diagonal if the `uplo` corresponds to the non-stored part. This makes `copytrito!` involving a triangular matrix equivalent to that involving its parent if the filled part is copied, and O(N) otherwise. Examples of improvements in performance: ```julia julia> using LinearAlgebra julia> A1 = UpperTriangular(rand(400,400)); julia> A2 = similar(A1); julia> @Btime copytrito!($A2, $A1, 'U'); 70.753 μs (0 allocations: 0 bytes) # nightly v"1.12.0-DEV.1657" 26.143 μs (0 allocations: 0 bytes) # this PR julia> @Btime copytrito!(parent($A2), $A1, 'U'); 56.025 μs (0 allocations: 0 bytes) # nightly 26.633 μs (0 allocations: 0 bytes) # this PR ```
Since `Base.@show` is much useful than `Base.Compiler.@show`.
Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: 9f8e11a4c New commit: 7b759d7f0 Julia version: 1.12.0-DEV Pkg version: 1.12.0 Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@9f8e11a...7b759d7 ``` $ git log --oneline 9f8e11a4c..7b759d7f0 7b759d7f0 Automatically upgrade empty manifest files to v2 format (#4091) 69c6de019 Remove duplicated word "different" (#4088) 87a4a9172 Actually switch to "Resolving Deltas" (#4080) ef844e32f Update CHANGELOG.md: link to [sources] PR (#4084) e10883ce5 REPLExt: check for compliant repl mode during repl init (#4067) ``` Co-authored-by: Dilum Aluthge <[email protected]>
This changes our IR representation to use a CodeInstance directly as the invoke function target to specify the ABI in its entirety, instead of just the MethodInstance (specifically for the rettype). That allows removing the lookup call at that point to decide upon the ABI. It is based around the idea that eventually we now keep track of these anyways to form a graph of the inferred edge data, for use later in validation anyways (instead of attempting to invert the backedges graph in staticdata_utils.c), so we might as well use the same target type for the :invoke call representation also now.
This was missed from #56299.
The return value of the LLVM instruction `fptosi` (https://llvm.org/docs/LangRef.html#fptosi-to-instruction) does not guarantee that the truncation of `NaN` is 0, so we relax the test to only check that the output has the expected type. Fix #56582.
Makes `test Compiler` work properly (as in use the Compiler package, not Base.Compiler) and pass tests, but still needs to be made parallel in a follow-on. --------- Co-authored-by: Shuhei Kadowaki <[email protected]> Co-authored-by: Kristoffer Carlsson <[email protected]> Co-authored-by: Shuhei Kadowaki <[email protected]>
These builtins are now special-cased within `abstract_call_known` after #56299, making them unnecessary for basic inference. As a result, their tfuncs have been removed in the PR. However the algorithm for calculating inlining costs still looks up these tfuncs, so they need to be recovered. Additionally, the `generate_builtins.jl` script in JuliaInterpreter also uses these tfuncs, so it would be worthwhile to register even simple placeholder tfuncs for now. @nanosoldier `runbenchmarks("inference", vs=":master")`
This is an alternative to #56532 and can resolve #31909. Currently `apply_type_tfunc` is unable to handle `Union`-argtypes with any precision. With this change, `apply_type_tfunc` now performs union-splitting on `Union`-argtypes and returns the merged result of the splits. While this can improve inference precision, we might need to be cautious about potential inference time bloat. --------- Co-authored-by: Jameson Nash <[email protected]>
With #56632, Compiler.jl as the stdlib can now be tested. However, the PR was incomplete, and when tests are actually run on `Compiler`, which is `!== Base.Compiler`, various errors occur, including issues caused by #56647. This commit resolves all these issues: - manage the code for loading `Compiler` in `setup_Compiler.jl`, ensuring that the stdlib version of `Compiler` is loaded when `@activate Compiler` is used beforehand - replace `Base.IRShow` with `Compiler.IRShow` - test `Base.Compiler.return_type` instead of `Compiler.return_type` This was split off from #56636.
External stdlibs that want to use this might not want to have to load REPL etc. REPL is used in https://github.com/JuliaLang/julia/blob/4709b6c48e79f6226e6dbee1b49bf7e563058ff7/test/runtests.jl#L215.
…st (#56586) Moved some `let...end` blocks into `@testset begin ... end` format. Added a test for converting a string to `Array{UInt8}`. Restored a commented out testset.
Should get the lazy strings file to 100% coverage
Codecov shows these as not covered yet
Only temporarily root objects during codegen, so that it is the responsibility of the caller to ensure the values live (including write barriers and old generations) only as long as necessary for correct execution, and not preserve values that never make it into the IR.
…ative (#56621) Was e.g. ``` ┌ Warning: Circular dependency detected. Precompilation will be skipped for: │ Base.PkgId(Base.UUID("eb0c05c4-6780-5852-a67e-5d31d2970b9a"), "ArrayInterfaceTrackerExt") │ Base.PkgId(Base.UUID("f517fe37-dbe3-4b94-8317-1923a5111588"), "Polyester") │ Base.PkgId(Base.UUID("0d7ed370-da01-4f52-bd93-41d350b8b718"), "StaticArrayInterface") │ Base.PkgId(Base.UUID("6a4ca0a5-0e36-4168-a932-d9be78d558f1"), "AcceleratedKernels") │ Base.PkgId(Base.UUID("244f68ed-b92b-5712-87ae-6c617c41e16a"), "NNlibAMDGPUExt") │ Base.PkgId(Base.UUID("06b0261c-7a9b-5753-9bdf-fd6840237b4a"), "StaticArrayInterfaceStaticArraysExt") │ Base.PkgId(Base.UUID("21141c5a-9bdb-4563-92ae-f87d6854732e"), "AMDGPU") │ Base.PkgId(Base.UUID("9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"), "Tracker") └ @ Base.Precompilation precompilation.jl:511 ``` Now  Thanks to @topolarity figuring out proper cycles tracking. --------- Co-authored-by: Cody Tapscott <[email protected]>
The text `"#= circular reference @-$d =#"` is printed yellow. Adds a test with the context `:color => true`.
This moves out LinearAlgebra into its own repo https://github.com/JuliaLang/LinearAlgebra.jl. This repo is still a bit bare (README needs to be added) but it has CI set up to run on buildkite (https://buildkite.com/julialang/linearalgebra-dot-jl/builds/18) and doc building on GHA. The external repo has all commits up to 4709b6c included in it. The reason for the move is to be able to focus issues and PRs and development regarding LinearAlgebra in one place.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )