Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cygwin build issue, libusb related? #906

Closed
T3sl4co1l opened this issue Mar 20, 2022 · 34 comments
Closed

Cygwin build issue, libusb related? #906

T3sl4co1l opened this issue Mar 20, 2022 · 34 comments
Labels
invalid This doesn't seem right

Comments

@T3sl4co1l
Copy link

I followed the build instructions as best I could (they aren't specific about which versions to enable in the package manager; apparently it's the -devel packages with include files) and, whether using build.sh or the cmake strings instructed, the build crashes and burns on dfu.c. I have:

-- Configuration summary:


-- DO HAVE libelf
-- DO HAVE libusb
-- DO HAVE libusb_1_0
-- DO HAVE libhidapi
-- DON'T HAVE libftdi
-- DO HAVE libftdi1
-- DISABLED doc
-- DISABLED parport
-- DISABLED linuxgpio
-- DISABLED linuxspi

and then from the build command,

...
[ 24%] Building C object src/CMakeFiles/libavrdude.dir/dfu.c.o
In file included from /cygdrive/e/MinGW/avrdude-6.4/avrdude/src/dfu.h:28,
from /cygdrive/e/MinGW/avrdude-6.4/avrdude/src/dfu.c:33:
/usr/include/w32api/usb.h:27:9: error: unknown type name ‘PVOID’
27 | typedef PVOID PIRP;
| ^~~~~

and a million other errors after there. I'm guessing it wasn't supposed to read the w32api include. But what else could it read? There's only one usb.h to choose from, right?

Running build.sh, I currently get the error:

Consolidate compiler generated dependencies of target libavrdude
[ 1%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi.c.o
In file included from /cygdrive/e/MinGW/avrdude-6.4/avrdude/src/avrftdi.c:41:
/cygdrive/e/MinGW/avrdude-6.4/avrdude/src/avrftdi_private.h:10:12: fatal error: libusb.h: No such file or directory
10 | # include <libusb.h>
| ^~~~~~~~~~

This after editing the cmake cache, which seems like a bad idea to me, but I saw this recommended elsewhere, and apparently it doesn't autodetect changes in missing files?

@dl8dtl
Copy link
Contributor

dl8dtl commented Mar 20, 2022

Unfortunately, both libusb 0.1 and Win32 libusb appear to use the same usb.h header file name.
So the order of the include directories is important, in order to pick the correct file here.
I'm absolutely no CMake expert (maybe @mariusgreuel can have a look at it), but maybe the following change would reorder the directories appropriately?

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index a8e0262..89f2146 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -226,6 +226,7 @@ set_target_properties(libavrdude PROPERTIES
     )
 
 target_include_directories(libavrdude
+    BEFORE
     PUBLIC
     "${PROJECT_SOURCE_DIR}"
     "${PROJECT_BINARY_DIR}"

@T3sl4co1l
Copy link
Author

Related question: would it make sense / be more maintainable / portable to include these libraries in the project (whether binaries or source; preferably source, for obvious reasons)?

  • Most (all?) are freely licensed
  • Even if not a compatible license, they're freely /available/, so could be added to subfolders with suitable (git?) commands
  • Several are obsolete / unmaintained / abandoned
  • Including them would greatly streamline the build (no optional libraries, build the best version, every time, anywhere?)
    I know very little about FOSS, or GNU, or non-GNU as the case may be, practices and norms, but every attempt I've made working with more-than-trivial projects seems to be a constant fight with version differences, package managers and dependency hell. I'm sure I'm doing something wrong, but I also see no obvious indication of what, each time. :(

@mariusgreuel
Copy link
Contributor

Its a little hard to troubleshoot from here, but it looks like you are running into a header naming problem between libusb and the Windows SDK, which both ship usb.h. For this reason, on Windows, the name for the libusb0 header is often lusb0_usb.h instead. Perhaps you can temporarily rename /usr/include/w32api/usb.h?

Also, you may ask the libusb team for help. I am curious myself how they think this is supposed to work.

I have not tried the Cygwin build in a while, but it did work for me the last time I worked on this. Perhaps the include order is different on my PC. I will try it some other day to see if it still works. That said, Cygwin is not my priority, as native builds work well on Windows.

Also, one common mistake is that people mix build systems. The automake build just puts intermediate files in the src folder, which screws up the CMake build. So, make sure you have a clean repo: git clean -xdf

would it make sense / be more maintainable / portable to include these libraries in the project

No. It may help you in this specific case, but libraries are a good thing.

@dl8dtl
Copy link
Contributor

dl8dtl commented Mar 22, 2022

For this reason, on Windows, the name for the libusb0 header is often lusb0_usb.h instead.

Does the CMake logic search for that header file? The autoconf scripts did consider it (but arguably, probably also in the wrong order - usb.h first, and only then lusb0_usb.h). usb_libusb.c requires just one of them only.

@T3sl4co1l
Copy link
Author

That seems to be the case. I got a successful build by:

Do ahead of cmake commands:
rn /usr/include/usb.h /usr/include/lusb0_usb.h

Change in dfu.h, micronucleus.c, pickit2.c, usb_libusb.c, usbasp.c, usbtiny,c:
#if defined(HAVE_USB_H)
#include <usb.h>
#elif defined(HAVE_LUSB0_USB_H)
#include <lusb0_usb.h>
#else
#error "libusb needs either <usb.h> or <lusb0_usb.h>"
#endif

to just

#include <lusb0_usb.h>

Obviously, easier solved by resolving -DHAVE_LUSB0_USB_H in the cmake, but I don't know cmake.

Or, I suppose renaming w32api/usb.h wouldn't have needed the extra editing. Oops. And moving root in front of /w32api in the include order should seem to work, too.

Is it... bad? to rename package files like this??

@mariusgreuel
Copy link
Contributor

Does the CMake logic search for that header file?

Yes. We have

check_include_file(usb.h HAVE_USB_H)
check_include_file(lusb0_usb.h HAVE_LUSB0_USB_H)
check_include_file(libusb.h HAVE_LIBUSB_H)
check_include_file(libusb-1.0/libusb.h HAVE_LIBUSB_1_0_LIBUSB_H)

That said, AFAIK, lusb0_usb.h is only used by libusb-win32. I documented the different combinations in the table found here: https://github.com/avrdudes/avrdude/wiki/The-story-on-libusb-for-Windows

but arguably, probably also in the wrong order - usb.h first, and only then lusb0_usb.h

Possibly. check_include_file, or better, the include folders and order baked into the compiler are still a mystery to me. Sometimes the wrong usb.h is picked up, sometimes not.

@mcuee
Copy link
Collaborator

mcuee commented May 8, 2022

I am one of the admins of libusb, libusbk and libusb-win32 (not the developer but on support and testing side). I agree that WinUSB (with MS OS descriptor) and libusb-1.0 API is the way to go for USB Debuggers/Programmers.

But I just want to clarify a few points.

  1. It is not complete true that libusb-win32 is not maintained -- even though we do not have formal release yet but libusb-win32 1.2.7.3 snapshot is signed and fixed some issues of libusb0.sys. libusb0.sys is rather stable but the problem is in the integration of libusb0.sys into libusb WIndows backend.

Ref: libusb-win32 1.2.7.3 was released on 13-Nov-2021 with signed kernel driver libusb0.sys
https://github.com/mcuee/libusb-win32/releases/tag/snapshot_1.2.7.3

  1. libusbk is also not unmaintained, even though the development of libusbk.sys has kind of stalled due to the certificate issue.

Ref: libusbk 3.1.0.0 was released on 19-Jul-2021 even though the kernel driver libusbk.sys was still the old 3.0.7 release.
https://github.com/mcuee/libusbk/releases/tag/V3.1.0.0
mcuee/libusbk#26

  1. libusb 1.0.25/1.0.26 release has fixed some issues related to libusbk.sys and libusb0.sys. So libusb0.sys may be working now for avrdude and many projects. Even though it is probably safer to use WinUSB as of now, it will be great that contributors can help to make the libusb0.sys support under libusb Windows backend better.

  2. I will recommend that avrdude to ditch the legacy libusb-0.1 API altogether (including libusb-win32) and move to libusb-1.0 API.

@dl8dtl
Copy link
Contributor

dl8dtl commented May 8, 2022

4. I will recommend that avrdude to ditch the legacy libusb-0.1 API altogether (including libusb-win32) and move to libusb-1.0 API.

Unlikely to happen. libusb-0.1 "just works", and it would need volunteers for such a change, to rewrite a lot of code – and test all this in the end, on all supported platforms, with all affected programmers. In the end, doing all that work for no gain other than replacing one API by another one, i.e. there won't be any improvements for end-users by that change.

@mcuee
Copy link
Collaborator

mcuee commented May 8, 2022

If the decision is to keep libusb-0.1 API, then the other possibility is to ditch libusb-win32 and use libusb-compat-0.1 under Windows.

@mcuee
Copy link
Collaborator

mcuee commented May 8, 2022

BTW I do not see the point of using Cygwin to build avrdude.

There is no issue to build native 32bit/64bit avrdude under MSYS2 mingw32/mingw64. MSYS2 has the libusb-compat-0.1 formula built-in.

@T3sl4co1l
Copy link
Author

@mcuee Nothing more than, I already had GCC 10 installed there, and saw Cygwin "recommended" on the Wiki.

@dl8dtl
Copy link
Contributor

dl8dtl commented May 8, 2022

Well, one question above was whether changing the include instructions in CMakeLists.txt would help to get the Cygwin build find the correct usb.h file.

@mariusgreuel
Copy link
Contributor

@mcuee I do not understand how a Windows source file that does a #include <usb.h> can differentiate between the Windows SDK header file and the libusb header file. Seems like pure luck which one you get.

In the context of building on Windows, wouldn't it be a good idea to rename (or alias) libusb's usb.h as libusb-win32 does?

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

@mcuee I do not understand how a Windows source file that does a #include <usb.h> can differentiate between the Windows SDK header file and the libusb header file. Seems like pure luck which one you get.

In the context of building on Windows, wouldn't it be a good idea to rename (or alias) libusb's usb.h as libusb-win32 does?

I do not know much about CMake, but there is no issue building under MSYS2 MinGW64 or MinGW32 with either libusb-win32 or libusb-compat-0.1 installed.

  1. MSYS2 MinGW64 build with libusb-compat-0.1.
    I belive in this case, pkg-config is used to detect libusb-compat-0.1.
MINGW64 /c/work/avr/avrdude
$ pacman -S mingw-w64-x86_64-libusb-compat-git
...

$ pkg-config -cflags libusb
-IC:/msys64/mingw64/include/libusb-compat -IC:/msys64/mingw64/include/libusb-1.0

$ pkg-config -libs libusb
-LC:/msys64/mingw64/lib -lusb

$ ./build.sh
-- Building for: Ninja
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/msys64/mingw64/bin/cc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/msys64/mingw64/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/msys64/usr/bin/git.exe (found version "2.36.0")
-- Found FLEX: C:/msys64/usr/bin/flex.exe (found version "2.6.4")
-- Found BISON: C:/msys64/usr/bin/bison.exe (found version "3.8.2")
-- Looking for libelf.h
-- Looking for libelf.h - found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - found
-- Looking for usb.h
-- Looking for usb.h - not found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for hidapi/hidapi.h
-- Looking for hidapi/hidapi.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DO HAVE    libhidapi
-- DON'T HAVE libftdi
-- DO HAVE    libftdi1
-- DISABLED   doc
-- DISABLED   parport
-- DISABLED   linuxgpio
-- DISABLED   linuxspi
-- ----------------------
-- Configuring done
-- Generating done
-- Build files have been written to: C:/work/avr/avrdude/build_mingw64_nt-10.0-22000
[62/62] Linking C executable src\avrdude.exe

Build succeeded.

Run

sudo cmake --build build_mingw64_nt-10.0-22000 --target install

to install.

  1. MSYS2 MinGW32 build with libusb-win32 installed. There is no pkg-config configuration for libusb-win32, so the header file detection is used ( lusb0_usb.h).
MINGW32 /c/work/avr/avrdude
$ pacman -S mingw-w64-i686-libusb-win32
...

$ pkg-config.exe --cflags libusb
Package libusb was not found in the pkg-config search path.
Perhaps you should add the directory containing `libusb.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libusb', required by 'virtual:world', not found

$ ls -la /mingw32/lib/pkgconfig/libusb*
-rw-r--r-- 1 xiaof xiaof 314 Apr 11 23:32 /mingw32/lib/pkgconfig/libusb-1.0.pc

$ ./build.sh
-- Building for: Ninja
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/msys64/mingw32/bin/cc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/msys64/mingw32/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/msys64/usr/bin/git.exe (found version "2.36.0")
-- Found FLEX: C:/msys64/usr/bin/flex.exe (found version "2.6.4")
-- Found BISON: C:/msys64/usr/bin/bison.exe (found version "3.8.2")
-- Looking for usb.h
-- Looking for usb.h - not found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for hidapi/hidapi.h
-- Looking for hidapi/hidapi.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Configuration summary:
-- ----------------------
-- DON'T HAVE libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DO HAVE    libhidapi
-- DON'T HAVE libftdi
-- DO HAVE    libftdi1
-- DISABLED   doc
-- DISABLED   parport
-- DISABLED   linuxgpio
-- DISABLED   linuxspi
-- ----------------------
-- Configuring done
-- Generating done
-- Build files have been written to: C:/work/avr/avrdude/build_mingw32_nt-10.0-22000
[62/62] Linking C executable src\avrdude.exe

Build succeeded.

Run

sudo cmake --build build_mingw32_nt-10.0-22000 --target install

to install.

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

Based on the above, I belive there is no issue now at least under MSYS2.

  1. libusb-compat-0.1 or libusb-0.1 under Linux or macOS or Windows MSYS2 will have the libusb.pc pkg-config script installed.
  2. libusb-win32 does not come with libusb.pc under Windows (and MSYS2 follows this) but the header file name is changed.

Last time Travis (libusb-win32 previous developer, current libusbk developer) changed the libusb-win32 usb.h header file name to prevent the conflict with Windows DDK usb.h file.

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

@mcuee I do not understand how a Windows source file that does a #include <usb.h> can differentiate between the Windows SDK header file and the libusb header file. Seems like pure luck which one you get.

In the context of building on Windows, wouldn't it be a good idea to rename (or alias) libusb's usb.h as libusb-win32 does?

You can see there is no issues under MSYS2. If there is an issue under Cygwin, I am not using Cygwin myself so I have no comments there.

However, it seems to come with libusb.pc file. So I think it should work as well if you use pkg-config.
https://cygwin.com/packages/x86/libusb-win32-src/libusb-win32-1.2.6.0-2-src
https://cygwin.com/git-cygwin-packages/?p=git/cygwin-packages/libusb-win32.git

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

BTW, cygwin listed libusb-1.0 and libusb-win32 as ORPHANED.
https://cygwin.com/packages/summary/libusb1.0-devel.html (still the very old libusb-1.0.21 version).
https://cygwin.com/packages/summary/libusb-win32-src.html

And the version of libftdi1 and hidapi are pretty old.
https://cygwin.com/packages/summary/libftdi1-devel.html (still the old version 1.4-2 released in 2020)
https://cygwin.com/packages/summary/hidapi-src.html (still the very old 0.8.0-rc1-1 released in 2015)

Of course you can build these dependencies by yourself but I really do not see any advantages of using Cygwin compared to MSYS2 mingw32/mingw64 build.

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

@mcuee Nothing more than, I already had GCC 10 installed there, and saw Cygwin "recommended" on the Wiki.

@T3sl4co1l

  1. Hmm, not really true. The avrdude wiki recommends Visual C.
    https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-Visual-Studio

On Windows, using Microsoft Visual C from Visual Studio is the recommended way to build AVRDUDE.

  1. MSYS2 and Cygwin are not really "recommended" by the Wiki.
    https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-MSYS2

If you prefer to use MSYS2 as your development environment, you may build AVRDUDE directly in MSYS2.

https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-Cygwin

If you prefer to use Cygwin as your development environment, you may build AVRDUDE directly in Cygwin.

  1. If we compare MSYS2 and Cygwin, I really do not see any benefits of using Cygwin, it is much slower than MSYS2 and do not really produce native Windows binary unless you use cross-build. And the avrdude dependencies are either not maintained or of old versions.

@mcuee
Copy link
Collaborator

mcuee commented May 9, 2022

@mariusgreuel

In the context of building on Windows, wouldn't it be a good idea to rename (or alias) libusb's usb.h as libusb-win32 does?

Just want to make it clear.

  1. libusb-compat-0.1 uses usb.h header file name but you can use pkg-config to detect it
  2. libusb-win32 does not use pkg-config but it has already renamed the header file name as lusb0_usb.h

BTW, by libusb we normally mean "libusb-1.0" as of now.

Another thing, I see that you have local fork of libftdi (https://github.com/avrdudes/libftdi) and hidapi (https://github.com/avrdudes/libftdi). I understand you have the local fork because of Visual Studio build (https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-Visual-Studio). Still you may want to get the changes integrated upstream. libftdi is anyway using CMake. hidapi has also moved to CMake.

As for your local fork of libusb-win32 (https://github.com/avrdudes/libusb), I am not so sure we will adopt CMake. I will say probably not as the official build method is still Windows DDK/WDK. And we have moved quite a bit far from libusb-1.0.26 release. So you may still want to keep the local fork.
Ref: https://github.com/mcuee/libusb-win32/commits/master

It will also be greatly appreciated that you update a bit on the wiki page on libusb (https://github.com/avrdudes/avrdude/wiki/The-story-on-libusb-for-Windows). It is mostly correct but I just want to point out that libusb-win32 and libusbk are not really unmaintained. The other thing is that under Linux there is two options for libusb-0.1 (mostly libusb-compat-0.1, but some distros like Debian/Ubuntu may still have the legacy libusb-0.1 due to some corner cases problems of libusb-compat-0.1). You can also use libusb-compat-0.1 under Windows (but may be a bit problematic if using Visual C but no problem under MSYS2).

Ref: Debian/Ubuntu bug report to replace libusb-0.1 with libusb-compat-0.1
https://bugs.launchpad.net/ubuntu/+source/libusb/+bug/991004
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731424
libusb/libusb#237 (one corner case for libusb-compat-0.1)

@T3sl4co1l
Copy link
Author

Exactly, "recommended" in scare quotes. (Should I have used double double quotes?..)

Notably, the Wiki doesn't say "NOTE: Cygwin does not build fully because XXX", so I saw no reason not to.

Anyway, I got my build, it works, and it is portable by the way, as long as you include all the dependencies (to wit: cygelf-0.dll, cygftdi1-2.dll, cyghidapi-0.dll, cygiconv-2.dll, cygintl-8.dll, cygncursesw-10.dll, cygreadline7.dll, cygusb-1.0.dll, cygusb0.dll, cygwin1.dll). I also since found @mariusgreuel 's release which Just Works(R) (and doesn't have the quirk that you have to enter the port as /dev/ttySx :P ).

@mariusgreuel
Copy link
Contributor

mariusgreuel commented May 9, 2022

I just want to point out that libusb-win32 and libusbk are not really unmaintained

@mcuee I guess what I wanted to point out in the Wiki is that a Windows kernel driver is a critical component in terms of system stability and security, and that a community driven project cannot provide the support that Microsoft can provide for WinUSB. libusb0.sys was great at the time it was created, however, somewhere between then and now, a transition from libusb0.sys to winusb.sys should have happend. However, v1.2.6.0 is still out there, very popular in the Arduino community, yet causing trouble. It certainly does no longer match my expectations, which is why I speak very critical of libusb0.sys in the Wiki. I did change the offending sentence slightly, though.

@dl8dtl
Copy link
Contributor

dl8dtl commented May 9, 2022

Well, a library is userland and thus not a kernel driver ... I'd also disagree about the support statement, based on experience, but I agree that "usb.h" was certainly not a good choice, being a too generic name.

Basically, libusb sits on top of kernel drivers, and all it does is to abstract the different OS driver APIs. That allows for a system-independent USB programming world. Without such an abstraction, each and every project that wants to use USB has to provide its own #ifdef spaghetti in order to handle multiple different OSes.

@mariusgreuel
Copy link
Contributor

Well, a library is userland and thus not a kernel driver

On Un*x, this is true. On Windows, libusb-win32 (which is now part of libusb) ships the kernel-mode driver libusb0.sys (and libusbK.sys), which is what I was talking about.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

I just want to point out that libusb-win32 and libusbk are not really unmaintained

@mcuee I guess what I wanted to point out in the Wiki is that a Windows kernel driver is a critical component in terms of system stability and security, and that a community driven project cannot provide the support that Microsoft can provide for WinUSB. libusb0.sys was great at the time it was created, however, somewhere between then and now, a transition from libusb0.sys to winusb.sys should have happened. However, v1.2.6.0 is still out there, very popular in the Arduino community, yet causing trouble. It certainly does no longer match my expectations, which is why I speak very critical of libusb0.sys in the Wiki. I did change the offending sentence slightly, though.

I totally understand your points in the following page even though I am not that happy with some of the words and some facts need to be corrected.
https://github.com/avrdudes/avrdude/wiki/The-story-on-libusb-for-Windows

  1. For AVRDUDE, we habe USB devices that use both WinUSB and libusb0.sys.

Correct.

  1. There is no one-fits-all library that supports the required libusb APIs and both libusb0.sys and WinUSB drivers: libusb-win32 supports libusb0.sys only, and libusb supports WinUSB and libusbK.sys only.

Incorrect.

a) libusb-win32 project libusb0.dll supports libusb0.sys and libusb.sys but not WinUSB.
b) libusbk project libusbk.dll supports libusb0.sys, libusb.sys and WinUSB.
c) libusb project libusb-1.0 supports libusb0.sys, libusb.sys and WinUSB. However, there may be some issues with the support of libusb0.sys (need to go with libusb-1.0.26 release, 1.0.24 release is problematic).

It will greatly appreciated people to test out and and report issues of using libusb-1.0.dll with libusb0.sys to libusb project.

  1. Requiring users to swap the vendor supplied driver to libusb0.sys is a no-go. Forcing a non-standard driver upon a device would require users to swap drivers back and forth depending on the tools used.

I understand this point. Still more experienced user can easily swap between libusb0.sys, libusbk.sys and WinUSB using Zadig.

  1. The library libusb-win32 should no longer be used, because it does not support WinUSB. If your USB devices uses the WinUSB driver, you should use the libusb library. If your USB devices use the Windows HID
    class driver, you should use libhidapi library.

I agree. I will highly recommend moving from libusb-0.1/libusb-win32 API to libusb-1.0 API.

  1. As libusb0.sys is outdated, buggy, and not supported, it should no longer be used. Instead, the Windows build-in driver WinUSB should be used.

Actually libusb0.sys is not really that buggy (main problem is integration of libusb0.sys support in libusb-1.0.dll) but yes I agree WinUSB should be the way to go.

libusb0.sys was developed much earlier than WinUSB.
libusbk.sys was developed to replace libusb0.sys and the motivation at the time was that WinUSB does not support isochronous transfer under Windows XP/Vista/7.
WinUSB finally supports isochronous transfer. So libusbk project shift the focus for better support of WinUSB.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

Well, a library is userland and thus not a kernel driver

On Un*x, this is true. On Windows, libusb-win32 (which is now part of libusb) ships the kernel-mode driver libusb0.sys (and libusbK.sys), which is what I was talking about.

libusb-win32 is a sperate project from libusb. It has the library part (libusb0.dll) and kernel driver part (libusb0.sys)
libusbk is a separate project from libusb. It has the library part (libusbk.dll) and kernel driver part (libusbk.sys)
libusb project is a pure cross platform library project.

Ref: https://github.com/libusb/libusb/wiki/Windows#about

This page details the specifics of the Windows backend part of libusb, which helps developers easily communicate with USB devices on Windows. Currently it supports the WinUSB and HID drivers for generic USB device access as well as the libusb-win32 and libusbK drivers.

Please note that libusb-win32 and libusbK are separate projects. libusb-win32 is a Windows-only project which provides a libusb-0.1 API compatible library for Windows and the associated kernel driver libusb0.sys. libusbK is a Windows only project which provides a new set of API for Windows (supporting WinUSB, libusb0.sys and libusbk.sys) and kernel driver libusbK.sys.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

https://github.com/libusb/libusb/wiki/FAQ#libusbwin32_libusbK_and_libusb_project

== libusb-win32, libusbK and libusb project ==

Take note [https://sourceforge.net/p/libusb-win32/wiki/Home/ libusb-win32] and [http://libusbk.sourceforge.net/UsbK3/index.html libusbK] projects are separate projects and both of them use [https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel libusb-win32 mailing list] for technical support. Unlike libusb which is a cross-platform project, libusb-win32 and libusbK project are both Windows only project.

libusb-win32 project includes libusb0.sys (Windows WDM kernel driver in device driver mode or filter driver mode) and libusb0.dll (libusb-win32 API, library). libusb-win32 API is a superset of the libusb-0.1 API supported by libusb-compat. libusb0.dll supports device using libusb0.sys and libusbK.sys.

libusbK projects includes libusbk.sys (Windows KMDF kernel driver) and libusbK.dll (libusbK API, library). libusbK API is Windows only and libusbk.dll supports device using libusbK.sys, libusb0.sys and WinUSB.

libusb Windows backend can support device using libusbK.sys (and libusb0.sys driver -- not recommended due to issues) through libusbk.dll provided by libusbK project. If libusbk.dll is present, it will use libusbK.dll as the intermediate library to support device using WinUSB driver. If libusbK.dll is not present, then it will directly talk to device using WinUSB using WinUSB API. libusb Windows also supports device using generic HID driver or usbdk driver.

In summary, there is no Windows kernel driver involved in the libusb project. It is purely a user space library. It supports multiple kernel drivers.
...
In terms of support, WinUSB is first class supported driver for libusb project, followed by libusbk.sys, then usbdk/HID/libusb0.sys which may have more issues.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

@mariusgreuel
Yes I agree that the situation is pretty complicated. What you write in the following page is already one of the best outside libusb project to explain the situations but I would prefer to correct some incorrect info.
https://github.com/avrdudes/avrdude/wiki/The-story-on-libusb-for-Windows

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

The other thing is that libftdi-0.1 has not been maintained for quite some time (no updates since 23-Dec-2014). It will be good to move to libftdi-1.0 whenever possible.

For FTDI device, usually the vendor FTDI driver is provided, so it is anyway needed to use Zadig to switch drivers if one wants to use libftdi-0.1 or libftdi-1.0. So there is no benefit to use libftdi-0.1 at all.
http://developer.intra2net.com/git/?p=libftdi;a=shortlog;h=refs/heads/libftdi-0.x

@mariusgreuel
Copy link
Contributor

@mcuee The whole discussion about libusb* is getting a bit silly. What I wrote in the wiki is in the context of avrdude on Windows, and while not a 100% accurate, it is correct in that context: I say 'libusb0.sys is not maintained', you say 'maintained, but not updated in ten years'. Really, to an end-user like me it is the same thing. We can go on like this forever.

My Wiki article does not claim to be 'The definite libusb analysis'. My idea was to transport some useful background info for a frustrated avrdude Windows users with the intend of troubleshooting. Further, I want to steer people away from using the kernel-mode driver libusb0.sys, and instead make use of the WinUSB driver, without having to use tools such as Zadig.

@mariusgreuel
Copy link
Contributor

Leaving the OT, this thread is about the header conflict usb.h on Windows. Depending on the compilers include path order, we currently get the libusb-0.1 header, or the Windows SDK header. I am still unclear on how to fix this.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

No problem -- skipping the libusb disucssion. It is good to sort out the issue first.
I will suggest to use pkg-config if possible to detect libusb-compat-0.1, then detect lusb0_usb.h. In that case, the Windows SDK usb.h will not be in the picture.

@mcuee
Copy link
Collaborator

mcuee commented May 10, 2022

I think this is a better way:
Let the user to select the header file name and library file name for the dependencies libraries.

In that case, the user can easily select the library to use.

  1. header file (eg: usb.h from libusb-compat-0.1 or lusb0_usb.h from libusb-win32)
  2. static library (eg: libusb-1.0.a) or the dynamic library (eg: libusb-1.0.dll.a).

In this case, people will also have the freedom to use the library to use for the VS build, no need to follow this one. To me the following build instruction is very special and not seen in other projects using CMake.
https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-Visual-Studio

Ref: #955 (comment)

@mcuee
Copy link
Collaborator

mcuee commented May 29, 2022

@mariusgreuel Maybe this can be closed as the OP can build within Cygwin. The other issues have already been discussed in the other threads.

@mcuee
Copy link
Collaborator

mcuee commented Jun 20, 2022

Close this for now. Improvements on usb.h vs lusb0_usb.h detection can be discussed in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

4 participants