Skip to content

add precompilation statements, using SnoopPrecompile #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

t-bltg
Copy link

@t-bltg t-bltg commented Nov 26, 2022

Has something like this been considered to reduce latency ?
Obviously, I do not know the internals of PythonCall, and this would need more extensive testing, but I think you get the idea.

Xref #45, for JuliaPlots/Plots.jl#4542.

pr

julia> @time using PythonCall
  7.115765 seconds (3.18 M allocations: 213.703 MiB, 2.23% gc time, 86.78% compilation time: 1% of which was recompilation)

master

julia> @time using PythonCall
  9.773912 seconds (7.86 M allocations: 454.046 MiB, 4.12% gc time, 89.42% compilation time: 73% of which was recompilation)

@cjdoris
Copy link
Collaborator

cjdoris commented Dec 2, 2022

Thanks for the PR! Nice idea to precompile all those init functions.

I've tried SnoopPrecompile in the past and not had much luck. Indeed on my computer, this PR makes importing PythonCall 0.4 seconds slower - 3.9 seconds on main branch vs 4.3 seconds here. Admittedly the 2.6 second gain on your computer is probably worth a small regression on mine. It would be great (but hard, probably) to understand the difference.

I also need to consider the effect of running all those init functions at precompile time. It's not obvious it won't mess something up - the fact that you needed to comment a few out due to segfaults backs this up. Will need to dig deeper.

I'm also hoping that upcoming native code cacheing in Julia will just magically take care of this sort of thing. 🤞

@t-bltg
Copy link
Author

t-bltg commented Dec 3, 2022

Obviously this approach is quite unsafe in terms of pointers, yes.

I've observed these segfaults in JuliaPlots/Plots.jl#4566 when trying to precompile the pyplot or pythonplot backends to improve ttfp. A part of the initialization still has to be done at runtime since you still have to initialize the python interpreter somehow.

Thanks for the comments here, my initial goal was to reach parity with pyplot, and this was achieved after the discussions in #45. We'd have to be more robust in order to use a SnoopPrecompile approach. I think the native code caching will speed things up, but I still think it will segfault because of the pointer nullification.

Anyway, let's close this for now: it was a first naive tentative.

@t-bltg t-bltg closed this Dec 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants