WIP: Add windows build

[Jason23347]

Todos

• Make socket compatable
• Build with Openssl
• Test with wine on ubuntu
• Test on Windows (win10)
• Header checks and function checks in configure.ac should be modified
• pkg-config doesn't work

Known problems

• SSL certificate verification alwasy fail.

Build from this branch

Download and compile mingw

On ubuntu it is easy:

sudo apt install mingw-w64

Download and compile openssl against mingw

I chose 1.1.1 release on github
There was an article about building openssl against mingw on linux:
https://marc.xn--wckerlin-0za.ch/computer/cross-compile-openssl-for-windows-on-linux

cd /path/to/openssl
./Configure mingw shared --cross-compile-prefix=x86_64-w64-mingw32-

before making, 3 lines should be added to file include/openssl/x509v3.h, just after #define HEADER_X509V3_H:

#ifdef X509_NAME
#undef X509_NAME
#endif

Build Axel

autoconf -i
SSL_CFLAGS=-I/path/to/openssl/include \
SSL_LIBS="-L/path/to/openssl -lssl -lcrypto" \
./configure --host=x86_64-w64-mingw32 --disable-Werror

This attachment file is an exutable file of windows environment.

Closes #108

[Jason23347]

@ismaell, please take a look at the code I changed in tcp.c, http.c, etc. I'm not sure if these changes are acceptable. Also I have doubts whether if-condition #ifdef HAVE_SSL in compat-win32.h is acceptable.

[ismaell]

why is this necessary?

[Jason23347]

This macro defines Windows system version, 0x0600 for Windwos Vista . Case AI_ADDRCONFIG was used in tcp_connect and defined in ws2tcpip.h, minimum value of _WIN32_WINNT should be 0x0600, which is defined to 0x502 in _mingw.h by default.

[ismaell]

this should probably be handled by the configure script.

Just like it defines _DARWIN_C_SOURCE for Darwin / Mac OS X, it should define _WIN32_WINNT for Windows.
E.g.:

case "$host_os" in
...
*-mingw32)
    AC_DEFINE([_WIN32_WINNT], [0x0600], [Request NT 6 API to expose AI_ADDRCONFIG in WinSock2])
;;

This is specially important because the definition is conditional, and ultimately depends on the software to request the right API.

[ismaell]

The proper way to add support for windows would be to move the OS test to the beginning of the configure script and avoid running the posix-specifc function tests for windows, instead including a compatibility library right away with an drop-in implementation.

[ismaell]

• Can WSAEWOULDBLOCK be translated to EINPROGRESS? is it really functionally equivalent in the test in tcp_connect?
• Why can't we provide emulation for fcntl instead of implementing SET_SOCK_BLOCK?
• Wrapping read and write and translating the socket fds should be also possible, albeit more complicated, if it can be done sensibly I would go for that.

[Jason23347]

• Can WSAEWOULDBLOCK be translated to EINPROGRESS? is it really functionally equivalent in the test in tcp_connect?

According to windows doc, I think they are functionally equivalent. But I'm not sure where should I translate WSA errors to errno.

• Why can't we provide emulation for fcntl instead of implementing SET_SOCK_BLOCK?

Ok. I will do the rebase for it.

• Wrapping read and write and translating the socket fds should be also possible, albeit more complicated, if it can be done sensibly I would go for that.

send and recv can be used for different systems, I have a question that why read and write should be kept in tcp_read and tcp_write. SOCKET type is unsigned. so changes from "-1" to INVALID_SOCKET seems necessary.

[ismaell]

I'm not making changes to correct code just because Windows fails to do the right thing. It's up to the compatibility layer for windows to fix any platform differences, transparently.

[ismaell]

Does the following work?

sh configure --with-ssl=/path/to/openssl --host=x86_64-w64-mingw32

That would be better than specifying SSL_LIBS and SSL_CFLAGS.

[Jason23347]

Does the following work?

sh configure --with-ssl=/path/to/openssl --host=x86_64-w64-mingw32
```is

That would be better than specifying `SSL_LIBS` and `SSL_CFLAGS`.

I have read the code in configure.ac, it seems with_ssl can only be set to yes/no. Originally there's no need to specify SSL_LIBS and SSL_CFLAGS, pkg-config could take care of modules using the PKG_CHECK_MODULES macro. The reason why they were specified is that OpenSSL could not be found by pkg-config automatically cause I didn't do the "make install" when I built OpenSSL. And it is not difficult to build axel (MinGW) using pkg-config with the libssl.pc file in OpenSSL path.

In all, this is not a problem with Axel. However, it might be necessary to add descriptions of pkg-config to the building guide.

[ismaell]

Indeed, I know it's only needed when libssl is in a non-standard path... I thought I had implemented taking a path on that arg before... I'll do that.

[ismaell]

I think documenting all the little options you have to configure a package is a little bit overkill.

[Jason23347]

I was considering the condition that you want to build from the master branch instead of the release tarballs, in which case you might have difficulties configuring OpenSSL. But it becomes less necessary that documenting for win32 build if the --with-ssl option could have a path arg.

[ismaell]

#325

[ismaell]

@Jason23347 I've implemented it, but I'm not sure it works on Windows... you still need to install openssl/libressl somewhere, so that the .pc files are found at the right place, but not necessarily in a standard path.

[Jason23347]

The code in #325 worked fine if the suffix "/lib/pkg-config" had been removed. The file libssl.pc could be found right in the OpenSSL directory by default.
Now, I can make a win32 build on Ubuntu PC with command

./configure --host=x86_64-w64-mingw32 --disable-Werror --with-ssl=/home/jason/openssl-OpenSSL_1_1_1

@ismaell Should I do the rebase to move all my commits afterward current master HEAD?

[ismaell]

Does the make install on openssl produce different results when compiling for Windows? is /home/jason/openssl-OpenSSL_1_1_1 an installation path??

[Jason23347]

By default, the make install with copy the libssl.pc and openssl.pc file to /usr/local/lib/pkg-config just as usual. I wonder if there is a necessity about building Axel with OpenSSL built but not installed...

[ismaell]

I wonder if there is a necessity about building Axel with OpenSSL built but not installed...

No, that's always the wrong thing to do, you must install it somewhere, not necessarily a global place...

[Jason23347]

Ok, I got it. It works fine while the installation path is configured.

[ismaell]

I'm just going to go ahead and add the _WIN32_WINNT flag to configure, do you mind to check it works?

[Jason23347]

I don't know why but the variable $host_os is set to mingw32 and cannot match the pattern *-mingw32... It works fine if *-mingw32 is changed to mingw32.

[Jason23347]

Tonight I will do the rebase to change SET_SOCK_BLOCK back to fcntl. But I still have no idea about the header checking in configure.ac... Some of the header files and functions does not exist in mingw.
@ismaell How should I modify the AC_CHECK_* part?

[Jason23347]

@ismaell It seems all worked fine except the function check of nanosleep failed, which is not provided by MinGW. Should we implement the function or use other function instead?

[ismaell]

Tonight I will do the rebase to change SET_SOCK_BLOCK back to fcntl. But I still have no idea about the header checking in configure.ac... Some of the header files and functions does not exist in mingw.
@ismaell How should I modify the AC_CHECK_* part?

Hmm, because it's a non-POSIX system, it should perhaps be handled before calling AC_CHECK_FUNCS, like this:

# OS-specific issues
AS_CASE(["$host_os"],
 [*mingw32], [
   AC_LIBOBJ([nanosleep])
 ])

You can find an implementation in gnulib.

[avdaredevil]

Hey @Jason23347 @ismaell any headway on this? It'd be nice to get this in as axel initially used to be windows compatible. Thanks :)

[ismaell]

@avdaredevil in slowmotion... I don't use Windows. It is a good candidate for a bounty.

[avdaredevil]

How close is this PR to being done, maybe I can sink a Saturday on this? Also what type of tooling is required to build this project?

[ismaell]

@avdaredevil you'll need MinGW, OpenSSL, WinSock and a pthreads implementation for starters. Cross-compiling is probably easier than building on Windows, but you can try either way.

[ismaell]

I'm going to modify configure to take the OpenSSL installation prefix in a SSL_PREFIX variable in the near future.

[ismaell]

@avdaredevil BTW; there's a msys2 package you can use as a starting point.

[ismaell]

https://github.com/ismaell/MSYS2-packages/blob/master/axel/PKGBUILD

[ismaell]

Would this pull request result in a native Windows version? I saw the MSYS2 package, but that is not native Windows.

Yes, this is not about MSYS2 but MinGW, AFAIK mingw-w64 produces fully native binaries; but of course, MSYS2 includes MinGW and the tools you need to actually run the buildsystem.

What will not be supported (by me) is an alternative buildsystem specifically for Windows.

Also, it might get us halfway to enable building with cl.exe, if someone wants that, and as long as it doesn't require any MS-specific change. See cccl.

[ismaell]

I found an alternate tool that works with native Windows:
<...>

This is not a help forum...