Intel Arc Graphics Thread #476
Replies: 40 comments 132 replies
-
|
UPDATE:
Before Intel update it, the code needs to be modified like this to avoid reporting errors: - match operation:
- case "multiply":
- output[top:bottom, left:right] = destination_portion * source_portion
- case "add":
- output[top:bottom, left:right] = destination_portion + source_portion
- case "subtract":
- output[top:bottom, left:right] = destination_portion - source_portion
+ if operation == "multiply":
+ output[top:bottom, left:right] = destination_portion * source_portion
+ elif operation == "add":
+ output[top:bottom, left:right] = destination_portion + source_portion
+ elif operation == "subtract":
+ output[top:bottom, left:right] = destination_portion - source_portionOf course not modifying it doesn't seem to affect normal usage. |
Beta Was this translation helpful? Give feedback.
-
|
I added this fix to master: 476d543 |
Beta Was this translation helpful? Give feedback.
-
|
About Karras scheduler: Currently using it causes a -996 "uses-fp64-math" error, which may be fixed in the next IPEX release (intel/intel-extension-for-pytorch#285) Until then, can specify to use # comfy/samplers.py
- sigmas = k_diffusion_sampling.get_sigmas_karras(n=steps, sigma_min=self.sigma_min, sigma_max=self.sigma_max, device=self.device)
+ sigmas = k_diffusion_sampling.get_sigmas_karras(n=steps, sigma_min=self.sigma_min, sigma_max=self.sigma_max, device="cpu") |
Beta Was this translation helpful? Give feedback.
-
|
is it Linux only or can you use this on a win10 machine? |
Beta Was this translation helpful? Give feedback.
-
|
You should be able to use WSL, but I haven't tested it. |
Beta Was this translation helpful? Give feedback.
-
|
So again, sharing the good news. An XPU version of their Pytorch extension with Pytorch 2.0 support,
Hats off to ComfyUI for being the only Stable Diffusion UI to be able to do it at the moment but there are a bunch of caveats with running Arc and Stable Diffusion right now from the research I have done. As of the time of posting: 1.) Setup can still be complicated in some respects and can randomly not work because Intel is only really verifying for enterprise Linux distributions and Ubuntu. My base Linux Fedora 38 install has the RPMs installed but I get a cryptic error regarding the runtime saying 2.) From what I have read at intel/intel-extension-for-pytorch#398, there is no native Windows version for some wheels to get 3.) Intel Arc right now has a huge problem with allocating more than 4GB of VRAM in IPEX even though the card has more VRAM in the case of the A750/A770 according to intel/intel-extension-for-pytorch#325. This seems to have mitigations elsewhere in Intel's oneAPI stack like their OpenCL compute runtime where anything you compile, you can work your code to go around the restriction via some flags and changed code according to intel/compute-runtime#627. No workarounds, it seems, until someone gets Intel to fix it. 4.) Because of that, experimenting with other Stable Diffusion UIs, it seems like every once in a while, Arc will occasionally run out of VRAM if you decide to not use lower VRAM flags and throw a 5.) Intel has an equivalent to I'm not even sure if I'm sure that Intel GPU support will not a big priority at the moment for the project, given the big blocker at this point is the 4GB VRAM limit which needs to be fixed by Intel. But that being said, things should be working a lot better than it is at the moment, maybe not average user ready, but should be ready for any mildly technical person. I really want to play with ComfyUI more but I really don't want to restart the application server for every several image I might want to generate even or rolling a dice for SDXL to actually finish a workflow. But the Arc cards are strong. I managed to equal the Nvidia RTX 3070 Ti in certain Stable Diffusion workflows with my 16 GB Intel Arc A770 so I look forward to the future when things are more mature and all the stars are aligned. Also not sure what issues need to be open here but there is potentially 2-3 of them that could be made from my report. Edit: Added a caveat I forgot to mention and filled in some information and fixed some typos. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe it doesn't like the shifting from ram to vram, try running it with --highvram |
Beta Was this translation helpful? Give feedback.
-
|
That's really counterintuitive. But yeah, it works for single image generation, was able to generate 5 images in a row without crashing. It does still crash when I tried to enable a batch of 2 because I decided to push my luck and may still crash like the other UIs when IPEX decides to randomly stop working but maybe that is to be expected given what I mentioned above about VRAM allocation and etc. But yeah, I think that brings this from theoretically useful to possibly useful. I'll file the |
Beta Was this translation helpful? Give feedback.
-
|
@simonlui From what I found
Quote from the last link: |
Beta Was this translation helpful? Give feedback.
-
|
I followed you steps, only used the updated version and then I got this issue can you tell me whats wrong? |
Beta Was this translation helpful? Give feedback.
-
|
So this took me a bit of time, but I have the Docker image I used published here so hopefully someone can find it useful. Some things to note I've found while poking and experimenting with things. 1.) A lot of the issues are gone mentioned before by @kwaa months ago like the Karras scheduler not working, where it is working now even with the new dpmpp_3 schedulers, or noise issues if not using split attention which is gone for the most part unless your graphics driver has crashed too many times and a restart of the computer fixes that. Anyways, hope people have good success with it like I did. I might try and see why ipexrun is failing but for now, I am going to take a break. Edit: Added in a caveat I forgot to mention and fixed some typos. |
Beta Was this translation helpful? Give feedback.
-
|
In the event anyone else missed it: I seem to be missing the file that gets sourced from /opt/intel, too... may edit this post once I figure it out. |
Beta Was this translation helpful? Give feedback.
-
|
Because Arch only packages the basekit and disaggregates all the other toolkits into seperate components which is kinda confusing why. Also, opt/intel/oneapi/setvars.sh should be part of the basekit package as you can see here. You may need to do a reinstall of the basekit if you don't see the file. |
Beta Was this translation helpful? Give feedback.
-
|
I may update this guide in the near future. |
Beta Was this translation helpful? Give feedback.
-
|
Well, short story. After spend whole day to install Arch Linux on Windows 10 WSL (2 in my case). After all procedures, I stuck on
The installer reach a 87% and starts roll back all changes, because intel-oneapi-basekit not contain library libtbb.so.12 that needed to install oneAPI AI Analytics Toolkit v2023. The packgage that contains needed library is intel-oneapi-compiler-shared-runtime but it conflict with intel-oneapi-basekit. Because the istall procedure can't finish with intel-oneapi-basekit, I install intel-oneapi-compiler-shared-runtime. ComfyUI runs: [{user}@pc ArchLinux]$ But if i try to generate image I get an next error: As can see, I tries params like --force-fp16 --bf16-vae --lowvram --use-split-cross-attention --highvram but it's have no effect. So, models are loaded, but KSampler are crashes. And that's my dead end, because I have no idea what's going on. Not in ComfyUI itself, and also there more in Linux Arch. But I suppose this can be a kind of report. |
Beta Was this translation helpful? Give feedback.
-
|
Well, first off, there's no reason for you to use Arch Linux on WSL2 anyways, because you can pick Ubuntu and it's the only supported distribution that Intel supports for non-enterprise users anyways so using anything else is going to be painful and with WSL, it's a fool's errand because it won't gives you any additional benefits like speed and etc. like you may think it does. Secondly, that's what I made my Docker image for because it's very easy to mess up the install and there are a lot of chances to go wrong. Just on a cursory glance, you have the wrong package probably for TBB and the compiler runtime conflict is probably the issue. If you do want to fix the issue, you're going to have to reinstall everything correctly. |
Beta Was this translation helpful? Give feedback.
-
|
Doesn't work on Windows too, unfortunatelly. Do a step-by-step and get "missing startup.sh". Because this error is not specific, without code of error - kind of common for docker using - it's have no specific result in google. Fixing that case basing only on your tutorial not seems possible. |
Beta Was this translation helpful? Give feedback.
-
|
@roflosaur Sorry, it took a while to diagnose because I needed to set up a virtual machine to debug the issue you ran into but I found the root issue which was git was changing the line endings of the files based on OS where startup.sh needed to be tied to Unix line endings only. I have set up a .gitattributes to fix this. I would advise you to do a clean checkout again via deleting and cloning my repository again or if you know the command line, you can re-checkout the repository with the new .gitattributes applied. Thanks for reporting the issue. |
Beta Was this translation helpful? Give feedback.
-
|
Here is an error when build the app. I also downloaded and installed manually Intel One API Base Kit version 2023.1.0.47256 but useless. Can't build an app thus. |
Beta Was this translation helpful? Give feedback.
-
|
Welp, aren't you unlucky on your timing here? Intel has broken their GPG key for OneAPI repository and that means nothing from oneAPI can be downloaded from said repository. I unfortunately can't help you here since my image depends on downloading those packages fresh from Intel for updating reasons. So until they get their stuff together over there, you're stuck as am I. If you want to follow progress on when a fix might come, see this issue |
Beta Was this translation helpful? Give feedback.
-
|
I don't use Windows, but all the pieces should be together to run ComfyUI on Windows now without any horrible downsides like no AOT compilation or missing packages since a new unofficial Intel Extension for Python package has been released without needing any installation of external dependencies and bundling it all together. It remains to be seen whether there will ever be an official package that does this so this package is the best chance anyone has at actually using Arc on Windows natively. According to reports, it is a bit faster than WSL2 but slower than Linux native. The rough steps to do this should be roughly the same as the process outlined in the opening post for Linux minus platform-specific things. 1.) Make sure you install an Intel driver that is 4952 or newer. The latest driver can be found here python -m ensurepip --upgrade3.) Install git from here using the GUI installer or other means. cd <location where to place ComfyUI>
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI5.) Download non-official Intel Extension for Pytorch packages here and install the packages with pip. Assuming they are all put in the root of the ComfyUI repository, run this command line. pip install intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whlAgain, I will need to remind you this is unofficial and does have a degree of risk but it should be mostly safe. 6.) Finish the rest of the installation pip install -r requirements.txtAt this point, you should be done with the installation. To run ComfyUI each time from scratch, open a terminal/command prompt. Then run the following command lines replacing <> with your own input: cd <location of ComfyUI>
python main.py <Any extra ComfyUI arguments you want to use> |
Beta Was this translation helpful? Give feedback.
-
|
When I try to install this node modifications, it's destroy my ComfyUI build. Because just rolling back all changes was not possible, I completely wiped out Python and Anaconda (as I thought, because despite deleting entries from the registry in AppData\Roaming\Python\Python310\site-packages something strange was still going on), and then follow to next procedure. This is not "ture github way", but at least this can help to someone, who want to launch ComfyUI on Intel Arc for Windows.
Then go to step 9, if no error occured. If you have any problems at this step, such messages like: or See the image:
and go to: To figure out for commented names. If names have "~" at start - remove it. Goto step 8 (copy, enter, and wait to end): 8.
or instead last command, if you have less than 16 Gb memory:
At this point, if all works fine, you can use DDU again, and install latest Intel Arc drivers. Should to work. Also you can optional install WAS node suite, by running "install.bat" in the "ComfyUI\custom_nodes\was-node-suite-comfyui-main" folder. REMEMBER: any additions for ComfyUI can destroy your ComfyUI installation for Intel Arc, because them build for CUDA mostly part, and trying to install it can destroy your Python dependences, critical for ComfyUI working. Don't forget download models! |
Beta Was this translation helpful? Give feedback.
-
|
I used IPEX v2.1.10 but it's not working as expected can you tell me whats wrong? |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the late reply but you are using the wrong version of Python and you didn't read my 2nd instruction point properly.
Do not use the newest 3.11 or 3.12 and use version 3.10 with the non-official packages, this will hold until IPEX updates again but it does not work correctly with the latest ComfyUI and IPEX package at the moment. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, but I also used cp311 version of ipex, which is for cpython 3.11 right? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, which is the core root of the problem. You aren't using the modified packages I have suggested in my instructions and that has not changed from the older v2.0.110+xpu version packages someone else has provided as the newest version of IPEX v2.1.10 does not work with ComfyUI on any OS platform. |
Beta Was this translation helpful? Give feedback.
-
|
Reinstall in forced mode, needed dll in place, but still get this error. Can it be relevant to folder rights or upper or lower letter case in "user"? |
Beta Was this translation helpful? Give feedback.
-
|
I found out about what is causing it. It's a dependency issue with libuv. I will overhaul the tutorial based on that shortly. For you, you'll need to uninstall the oneAPI GUI installed components and then backtrack and follow steps 3 onwards but then the method I will provide should replace that in a much more easy manner. |
Beta Was this translation helpful? Give feedback.
-
|
There really needs to be a better, more updated tutorial for this. Much of the information is stretched out over the thread and mixed up with information between Linux and Windows |
Beta Was this translation helpful? Give feedback.
-
|
I'll consider writing one for Ubuntu because it's the only Linux distro on the consumer side supported officially by Intel unless you pay the big money for Red Hat and SUSE support with their enterprise GPUs. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, should to try too. |
Beta Was this translation helpful? Give feedback.
-
|
The whole thing about installing the OneAPI AI Analytics toolkit didn't seem necessary though. On a fresh install of Arch, All I did was install Torch 2.0.1 and intel-compute-runtime-bin from Arch repositories, then install the Intel extension for pytorch with the code below
And then from a terminal in the directory where ComfyUI is installed I used
And that line above I added to a start.sh script, which can include launch options for easier usage. But yeah, that was all I had to do and it takes me about 4 seconds to do 20 steps at 512x768. I'm not even sure I needed to get anything from Arch's repository, if so, those packages should be available in any repository, which would make the install fairly simple for any distro. |
Beta Was this translation helpful? Give feedback.
-
|
@burntbyhellfire If you don't use Intel Python, you don't need to install the AI Kit. |
Beta Was this translation helpful? Give feedback.
-
|
If we both have a GPU of the same specs we could both run some benchmark prompts and settings to see if there really are any performance improvements with doing it that way. I just got a Sparkle A770 16GB. |
Beta Was this translation helpful? Give feedback.
-
|
To install on Ubuntu. 1.) Install Linux drivers with the following instructions provided by Intel here sudo apt install python3-pip git3.) Install ComfyUI with the following terminal commands replacing the <> portion with a selection of your choice. cd <Location to put ComfyUI>
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI4.) Install all the Intel Extension for Pytorch pip Python packages first with this terminal command. python -m pip install torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.110+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/5.) Finish the rest of the dependency installation with this terminal command: pip install -r requirements.txtInstallation should be done at this point. To run ComfyUI, one can type in the following terminal commands replacing <> with your own input: cd <location of ComfyUI>
python main.py <Any extra ComfyUI arguments you want to use>For other Linux distros outside of Arch Linux or Ubuntu, one will need to manually install the Intel compute runtime, git, a Python with a version supported by Intel Extension for Pytorch like 3.10 as of the time of this writing or using Intel's Python from the AI Kit which will provide that, pip, and any other dependencies required according to your own Linux distribution's package manager or install scripts but one should be able to then just follow step 3 and onwards without any issue afterwards. |
Beta Was this translation helpful? Give feedback.
-
|
I've updated the Arch Linux setup guide for the latest IPEX and ComfyUI, without oneAPI AI Kit. If anyone wants to uninstall the previously installed AI Kit, it's available: cd /opt/intel/oneapi/installer
sudo ./installer --action remove --product-id intel.oneapi.lin.aikit.product --product-ver 2023.1.0-31760
# if you don't need
paru -Rsc libxcrypt-compat |
Beta Was this translation helpful? Give feedback.
-
New Patch!Thanks to vladmandic/automatic for the code. contributors Basically, you just need to copy Then modify try:
import intel_extension_for_pytorch as ipex
if torch.xpu.is_available():
xpu_available = True
+ from attention import attention_init
+ ok, e = attention_init()
except:
passIt could partially fix intel/intel-extension-for-pytorch#325. (so f**k you, intel) |
Beta Was this translation helpful? Give feedback.
-
|
That's more of a workaround, I've seen his code but I didn't think @comfyanonymous would enjoy special casing his code with a specific attention implementation that would work for Intel GPUs only for working around the issue which is why I didn't submit a variant it when I wrote the patches for enhanced IPEX support. Doing manual memory management like this is a pain in Python and doesn't look particularly nice. I think Intel has to fix it on their end, asking end users of IPEX to do this to every project that implements IPEX is untenable. |
Beta Was this translation helpful? Give feedback.
-
|
I couldn't find any performance gains with Ipex in tests I ran in ComfyUI before, and after setting up an environment and running Comfy from it. |
Beta Was this translation helpful? Give feedback.
-
|
interesting tho, that i didn't do anything with modules/intel/ipex/attention.py, but problem with 4GB limit is gone after patching p.s. error still continue to occur ^_^ |
Beta Was this translation helpful? Give feedback.
-
|
It seems that only older versions of |
Beta Was this translation helpful? Give feedback.
-
|
Cool... Got it working with WSL2 using Arch. Followed the instructions at the top, as well as installed jemalloc paru(root/native environment), openmp via pip(venv). There are issues with the env vars as Arch is pulling my windows env vars through. So I some times have to re run the setvars.sh. I get warnings about libpng when I run via python or ipexrun. Everything still works though. I generally run through ipexrun but add xpu to the command i.e.
Apart from some memory overruns I've had no issues the last few days. |
Beta Was this translation helpful? Give feedback.
-
|
anybody can tell where intel store their patches for pytorch? |
Beta Was this translation helpful? Give feedback.
-
|
You can find these "patches" here but it's really not that much different over just compiling the source tree provided. |
Beta Was this translation helpful? Give feedback.
-
|
thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Yes, and pytorchvision and other associated packages. |
Beta Was this translation helpful? Give feedback.
-
|
In an unusual but welcome move, Intel released a version of Intel Extension for Pytorch for XPU, v2.1.30+xpu, that has the fixes needed to run ComfyUI on Linux and Windows without code modifications, from what wheels I could see in the repository. I was able to verify SD 1.5 is working, but SDXL is being a bit wonky on my end so I need some further time to test it out and see what is up. Do note as well that Linux and the Intel Compute Runtime currently as of writing this has some outstanding issues that are being worked through with the latest versions like this which may be related to my issue so tread carefully. If anyone can verify that this can run ComfyUI without any changes, that would be great so I can move forward with creating a pull request for some code changes I have been meaning to merge into the project. Edit: I verified that SDXL is working for the most part but it is pretty unstable without VRAM reduction built in for higher resolution images and I am getting hit by the slower speed Intel generation speeds by the bug I mentioned above so it's best to stay on a kernel at 6.6.25 LTS/6.8.4 or lower if possible to avoid getting slowdowns at this time. Might change in the future. |
Beta Was this translation helpful? Give feedback.
-
|
Tested with SD1.5 and SDXL. Seems to work. Over 4GB allocations still fail without the attention.py fix. Stable cascade generated either black og garbled image |
Beta Was this translation helpful? Give feedback.
-
|
Stable Cascade and Wuerstchen IPEX issue is being tracked at intel/intel-extension-for-pytorch#529. Over 4GB allocations have been discussed here but the discussion I had months prior in oneapi-src/oneDNN#1638 suggests it needs to be fixed in hardware and Intel decided the hit to translate above 4GB memory addresses takes too much of a performance hit for their liking so Alchemist will probably never see this fixed but Battlemage and up might. |
Beta Was this translation helpful? Give feedback.
-
|
This seems to be the last working recipe for me (Arch Linux): #476 (comment) Since that posting the situation devolved significantly from running ComfyUI out of the box:
Am I holding it wrong or is that what it is at the moment? |
Beta Was this translation helpful? Give feedback.
-
IPEX is fine with the latest version, v2.1.30+xpu, as stated in the prior post to yours.
Correct, this is the only issue currently affecting things that people running rolling Linux distributions will hit, see intel/compute-runtime#726 . The issue is only fixed at the time of this writing in Intel's fork of the kernel in drm-intel-gt-next. And it gives way less performance in the fixed kernel at the moment, 1/4th according to the bug report. Best to stay either on 6.8.4 or 6.6.25 LTS or older if possible.
IPEX has clearly defined Python requirements in its installation page that guides you through the process. The latest Python Intel builds and supports is 3.11 so that is what I stick with in my Docker image and you may possibly want to use that if you don't want to bother with keeping track of and maintaining the correct versioning for the packages needed to run ComfyUI. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Core Ultra iGPU ？ |
Beta Was this translation helpful? Give feedback.
-
|
It's recommended to use upstream packages and install process if possible but yes, if I recall correctly, Meteor Lake has not been updated to be supported in upstream IPEX. |
Beta Was this translation helpful? Give feedback.
-
|
Does this also hold true for Win10? Is this still the case? "ComfyUI can run on Windows 11 natively without any flag finally! with the v2.1.20+xpu packages." I'm having tremendous issues getting my A770 16GB working with Comfyui. Does anyone have a current guide to install / setup? |
Beta Was this translation helpful? Give feedback.
-
|
Just reporting here that SD3 does indeed work, |
Beta Was this translation helpful? Give feedback.
-
|
I'm not sure if anyone else has been having this issue with Archlinux, but a fairly fresh OS install, following the usual instructions, creating a venv environment with python 3.11, installing the latest intel extension for pytorch, setting the setvars.sh, oneapi all installed, but go to run ComfyUI and I get an import error: libintel-ext-pt-gpu.so: undefined symbol: _ZNK4sycl3_V16detail16AccessorBaseHost25isMemoryObjectUsedByGraphEv |
Beta Was this translation helpful? Give feedback.
-
|
Would it be better to actually move to a Debian/Ubuntu based distribution for Stable Diffusion then given these issues and how Arch seems to have more issues than others with it not being targeted for any kind of official support? |
Beta Was this translation helpful? Give feedback.
-
The bug was fixed in all the Linux kernels supported if you had the correct compute runtime version around a month ago. But the performance regression didn't get fixed until recently with 6.9.4 for that particular bug. |
Beta Was this translation helpful? Give feedback.
-
|
I got it installed and working with 2.1.10+xpu for now and I'm going to try to get 2.1.30+xpu working using the command line installer for oneapi-basekit. But about this kernel issue. 6.9.5 is available in the repo, would it solve these regression issues to upgrade to this now? And all should be well? |
Beta Was this translation helpful? Give feedback.
-
|
6.9.4 and later will have it, so you will be fine if you use that. Again, the tricky part is making sure your oneAPI basekit install lines up with what IPEX wants. Good luck. |
Beta Was this translation helpful? Give feedback.
-
|
I used the offline installer script to install 2024.1.0, and IPEX 2.1.30+xpu. It's all running as normal, I'll update the kernel soon and I should be gucci. |
Beta Was this translation helpful? Give feedback.
-
|
Any experience wether intel is any good or cost effective in this regard? After buying my 4090, I want to experiment with other types of GPUs, because half price for same or more ram in next gen will make me veer towards other brands, because Nvidia is screwing us all with their failure to have done a 48GB Vram version of the 4090. |
Beta Was this translation helpful? Give feedback.
-
|
How can I use IPAdapter? |
Beta Was this translation helpful? Give feedback.
-
Did you solve it? I have the same issue. |
Beta Was this translation helpful? Give feedback.
-
|
并没有完全解决，临时的解决办法是：在图片后面加一个clip 视觉图像处理，确实可以工作，但是只能参考正方形的图片，很有局限性，如果你有其他的办法，或者有什么要交流的，可以发邮件[email protected] |
Beta Was this translation helpful? Give feedback.
-
|
你去intel官方comfyui群问的试试 https://discord.com/channels/554824368740630529/1193952640225267802 |
Beta Was this translation helpful? Give feedback.
-
I asked. the third to last message is mine actually. :D but no one responded. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Intel has released IPEX v2.1.40+xpu and oneAPI 2024.2 which solves some outstanding issues like one where VAE Decode would hang and segfault which some people had with a faulty package upgrade which the update also fixes. |
Beta Was this translation helpful? Give feedback.
-
|
I could run FLUX dev example workflow with fp8+fp8 75sec on windows (which is 10sec faster than RTX3060), until it uses shared GPU memory. I am going to try GGUF quants today. |
Beta Was this translation helpful? Give feedback.
-
|
I tested Q4 of GGUF, and it works good. I think you should use forge if you are Windows users. I hope Intel add option to disable shared GPU memory like nVidia, but not very hopeful. |
Beta Was this translation helpful? Give feedback.
-
|
With new option you may need more value for Previously, generation become very slow, when it starts using shared VRAM. Generation took 62.04sec(2.61s/it) for 1024x1024 20steps with fp8 of flux and t5-v1_1-xxl-encoder-Q5_K_M.gguf. |
Beta Was this translation helpful? Give feedback.
-
Current GGUF implementation for Flux is slower than FP8, and it may use more VRAM. |
Beta Was this translation helpful? Give feedback.
-
Not sure if to buy 16GB 4060ti or A770/RX7600 XT which are cheaper, the speed seems to be similar |
Beta Was this translation helpful? Give feedback.
-
4060ti 16gb would be much better to setup and easier to use, some custom nodes only work on nVidia, |
Beta Was this translation helpful? Give feedback.
-
I was able to get Comfyui running on my Arc A770 with the help of https://www.reddit.com/r/comfyui/comments/1ev7ym8/howto_running_flux1_dev_on_a770_forge_comfyui/ and runs with Flux as well. As for what one to get? I have never been happy with anything other than nVidia / CUDA for any AI related uses. Granted my experience with AMD and Stable Diffusion is nil. Given issues when it first came out, I never bothered with my AMD cards. My A770 performs great but very incompatible. Worked out of the box with SD.Next and Comfy was a challenge. Now that it's working, I still have issues with LivePortrait, and other video related models like AnimatedDiff and SVD. My 4060ti just chugs along with no problems. For me, there's no point going with anything other than nVidia unless you're getting it just for gaming. You're not saving anything because you're losing functionality and compatibility. |
Beta Was this translation helpful? Give feedback.
-
On Linux, it is slower but it is still in the same ballpark speedwise. The main issue I suspect here is the fact that Windows has no AOT enabled in is release build still other than the custom one I pointed it in an earlier comment that by now is months old. Fixing this would require custom building the IPEX package of software and for most people that is way above their heads and not feasible. |
Beta Was this translation helpful? Give feedback.
-
|
I noticed pytorch has some xpu nightlies for both linux and windows that can be installed with:
How well do these work? |
Beta Was this translation helpful? Give feedback.
-
|
I've yet to try as I am in Japan on vacation without access to my workstation for two more days but I had intended with my prior patch that it should work out of the box but I have a feeling some wrappers might be managed incorrectly so I'll have to see if those need to be rewritten in a torch compliant way. I don't know though how much of IPEX Intel has upstreamed though so performance could be crappy but I will report back once I have had time to test this. |
Beta Was this translation helpful? Give feedback.
-
|
@comfyanonymous thank you! Downloaded 2.6.0 torch does not work, the latest 2.5.0 in the Pytorch repository runs ComfyUI, to run it need to install intel-for-pytorch-gpu-dev and intel-pti-dev packages instead of OneAPI ToolKit following the recommendations of https://pytorch.org/docs/stable/notes/get_start_xpu.html. But it seems that need to change something in ComfyUI, because the following error appears-
python main.py --highvram --dont-upcast-attention --use-quad-cross-attention --preview-method none --fp8_e4m3fn-text-enc --bf16-vae --fp8_e4m3fn-unet To see the GUI go to: http://127.0.0.1:8188 Prompt executed in 2.10 seconds |
Beta Was this translation helpful? Give feedback.
-
|
This is what I feared. All the |
Beta Was this translation helpful? Give feedback.
-
|
@simonlui thanks, will be looking forward to it! I tried removing the lines related to ipex and I was able to load the checkpoint model, but got stuck at clip text encode with the error “RuntimeError: Native API failed. Native API returns: -30 (PI_ERROR_INVALID_VALUE) -30 (PI_ERROR_INVALID_VALUE)”, I managed to load only with --force-fp32, but now ksampler gives the same error. I hope you are able to solve this! 😄 |
Beta Was this translation helpful? Give feedback.
-
|
@comfyanonymous I have tested the nightly and on a basic level, it does work if you use the |
Beta Was this translation helpful? Give feedback.
-
|
The new This allowed for |
Beta Was this translation helpful? Give feedback.
-
|
I got an error. Are the versions in your command correct? Arch Linux. $ pip install torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.120+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl-aitools/ Looking in indexes: https://pypi.org/simple, https://pytorch-extension.intel.com/release-whl-aitools/ |
Beta Was this translation helpful? Give feedback.
-
|
To be honest, I had seen that and that Intel page, but it looked so complex and I could not understand, so I came here. Anyway, I tried two highest versions, but both failed. $ python -m pip install torch==2.3.1+cxx11.abi torchvision==0.18.1+cxx11.abi torchaudio==2.3.1+cxx11.abi intel-extension-for-pytorch==2.3.110+xpu oneccl_bind_pt==2.3.100+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ Looking in indexes: https://pypi.org/simple, https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ $ python -m pip install torch==2.1.0.post3 torchvision==0.16.0.post3 torchaudio==2.1.0.post3 intel-extension-for-pytorch==2.1.40+xpu oneccl_bind_pt==2.1.400+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ Looking in indexes: https://pypi.org/simple, https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ For what it's worth, the following packages have been installed on my computer.
|
Beta Was this translation helpful? Give feedback.
-
|
@HubKing, what's your version of python - 3.12? if so, then there is no whl package compiled for python 3.12 in official repos. |
Beta Was this translation helpful? Give feedback.
-
|
It's |
Beta Was this translation helpful? Give feedback.
-
|
That's kind of on you to handle. You either need to deal with it by installing an older Python version side by side your current one, and then manage a virtual environment for that version of Python or do something similar with |
Beta Was this translation helpful? Give feedback.
-
|
I asked ChatGPT for a way to run a different version of Python, and succeeded at least starting ComfyUI. Since a lot of Arch users would have the same problem, I will write the process in case other Arch users have the same problem in the future.
|
Beta Was this translation helpful? Give feedback.
-
SDXL Checkpoint Not Utilizing Intel Arc GraphicsHey everyone, I've been following the instructions from #intel-gpus and the installation guide for the platform GPU version v2.3.110+xpu on WSL2 (Ubuntu 24.04). My hardware setup includes an Intel® Core™ Ultra 9 185H and Intel® Arc™ Graphics on a Windows 11 laptop, running Python 3.11.10. I've installed the Intel® Arc™ & Iris® Xe Graphics - WHQL - Windows* 32.0.101.5768 and followed the installation guide for Intel's oneAPI Basekit without any issues. I also checked out the 950.13, but I passed on it since I'm using Ubuntu 24.04. According to the Installing Client GPUs guide, "The easiest option is to use the Ubuntu Desktop-supplied Hardware Enablement (HWE) starting with Ubuntu 23.04 for integrated and discrete graphics. This is the simplest option and does not require any custom settings." I installed the Python dependencies using the following command: python -m pip install torch==2.3.1+cxx11.abi torchvision==0.18.1+cxx11.abi torchaudio==2.3.1+cxx11.abi intel-extension-for-pytorch==2.3.110+xpu oneccl_bind_pt==2.3.100+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/After completing the virtual environment setup and installing the repository dependencies, I ran the sanity test and received the following output: Additionally, I then started ComfyUI using the following commands: source .venv/bin/activate
source /opt/intel/oneapi/setvars.sh
python main.py --use-pytorch-cross-attention --highvramThe initialization seemed to work fine, and the server started without any errors. However, when I ran an SDXL checkpoint, it did not utilize the Arc Graphics. Here is the complete server log: Any ideas on what might be going wrong? Thanks in advance for your help! |
Beta Was this translation helpful? Give feedback.
-
|
It is though? The log output indicates it is using GPU given the device type is XPU and other log lines. Are you somehow expecting SDXL generation to be super fast? I think your expectations a bit unfounded here. Even the Nvidia 4090 will need a couple of seconds to spin up an image. You can try and see if it gets any faster if you have enough RAM to offload everything to GPU via Edit: Also try giving the TorchCompileNode a try, it should just work without needing to do FP64 emulation and should speed up generation a bit. |
Beta Was this translation helpful? Give feedback.
-
|
@simonlui, |
Beta Was this translation helpful? Give feedback.
-
|
Hi @simonlui, After running the test script below, the problem magically disappeared. I guess it needed some warm-up. Very weird, but still, thanks for your quick response! import torch
import intel_extension_for_pytorch as ipex # noqa: F401
print(torch.xpu.is_available())
device = torch.device("xpu")
torch.random.manual_seed(42)
tensor_a = torch.randn(15000, 15000)
tensor_b = torch.randn(15000, 15000)
tensor_a = tensor_a.to(device)
tensor_b = tensor_b.to(device)
result = torch.matmul(tensor_a, tensor_b)
print(result) |
Beta Was this translation helpful? Give feedback.
-
|
The log messages are normal, not every model can be optimized with Intel's function call exactly so it spits it out. Not sure why your system needed that much of a warmup, the first generation is usually slower but not that slow to the tune of 512x512 taking 300 odd seconds. IPEX does ship with AOT compilation where it is already done but it is what it is probably for iGPUs. Glad to hear you got it working after having it warm up. |
Beta Was this translation helpful? Give feedback.
-
|
Unless you're running a very specific version of a NVIDIA graphics card on Linux with a very specific version of CUDA and have very specific software installed in a very specific order and way, forget about getting AI generated images to work on your computer. Ignore what you hear and see otherwise, it's not possible. Save yourself the time and headache of this massive software and hardware Cluster F... and just pay for a subscription on a website if you want to generate AI images. |
Beta Was this translation helpful? Give feedback.
-
|
AI applications in general are not easy to use quite yet for your average computer user, and it's understandable to be frustrated. A lot of these things even proclaiming one click compatibility can break and it's not quite clear how one can fix it if not technically inclined. However, it is out of scope here because this thread is explicitly trying to troubleshoot ComfyUI which is explicitly "batteries not included" and trying to make things possible. You can't expect ease to use to be prioritized given that. |
Beta Was this translation helpful? Give feedback.
-
I agree that it can be headache-inducing and time-consuming, but it's definitely not "impossible." I've been using ComfyUI smoothly with my Intel ARC A770 16GB graphics card for a long time now. When Flux Dev was released, I ran it without any issues and still use it to this day. So, it’s far from impossible—actually, it’s quite easy. The cheapest Nvidia card with 16GB on the market is the RTX 4060 Ti, which costs almost twice as much as the A770. However, there isn’t a such difference in performance. Therefore, for budget-conscious people who can troubleshoot issues, the A770 16GB is a viable option, and they can produce images without a problem. If we’re talking about the real challenge, it’s not image generation itself. The actual challenge lies in the extensions, nodes, etc. For instance, I can’t use Animatediff, and there are other extensions I can't access either. That’s why I plan to switch to an Nvidia GPU when my budget allows. However, if my only goal were standard image generation, I wouldn't feel the need to switch. The A770 is a very suitable card for that purpose. |
Beta Was this translation helpful? Give feedback.
and others
-
ComfyUI now supports Intel Arc Graphics. (#409)
Since the installation tutorial for Intel Arc Graphics is quite long, I'll write it here first.
Intel Extension for PyTorch is currently only available for Linux, so you will need to have a Linux or WSL environment.
Arch Linux (with
paru) are used here as example operating systems.Install Python and PIP:
Install Intel Compute Runtime and Intel oneAPI Base Kit:
Install ComfyUI:
Install Dependencies (via
venv):python -m venv venv source venv/bin/activate pip install torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.120+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl-aitools/ pip install -r requirements.txtFor the second start and beyond, venv needs to be reactivated:
source venv/bin/activateSet oneAPI vars:
source /opt/intel/oneapi/setvars.shRunning ComfyUI (via
python):Running ComfyUI (via
ipexrun):Beta Was this translation helpful? Give feedback.
All reactions