Changelog for the libssh2 project. Generated with git2news.pl Version 1.11.0 (30 May 2023) Daniel Stenberg (30 May 2023) - libssh2.h: bump to 1.11.0 for release GitHub (30 May 2023) - [Will Cosgrove brought this change] Libssh2 1.11 release notes, copyright (#1048) * Libssh2 1.11 release notes, copyright Viktor Szakats (29 May 2023) - add copyright/credits Closes #1050 - ci: add LIBSSH2_NO_AES_CBC to GNU Make build Closes #1049 - ci: add wolfSSL Linux builds Exclude wolfSSL builds from tests. All fail: ``` 2/43 Test #2: test_aa_warmup ............................***Failed 5.59 sec libssh2_session_handshake failed (-44): Unable to ask for ssh-userauth service ``` Ref: https://github.com/libssh2/libssh2/actions/runs/5085775952/jobs/9139583212#step:12:942 (with logging) Ref: https://github.com/libssh2/libssh2/actions/runs/5085586301/jobs/9139192562#step:12:225 wolfSSL version: ``` Get:1 http://azure.archive.ubuntu.com/ubuntu jammy/universe amd64 libwolfssl32 amd64 5.2.0-2 [818 kB] Get:2 http://azure.archive.ubuntu.com/ubuntu jammy/universe amd64 libwolfssl-dev amd64 5.2.0-2 [1194 kB] ``` Cherry-picked from #1046 Closes #1046 - ci: mbedTLS build config tidy-up Cherry-picked from #1046 - wolfssl: fix detection of AES-GCM feature Follow-up to df513c0128e1a811ad863d153892618e728845f0 Ref: https://github.com/libssh2/libssh2/issues/1020#issuecomment-1562069241 Closes #1045 - build: fix 'unused' compiler warnings with all `NO` options set - add `LIBSSH2_NO_ED25519` build-time option to force-disable ED25519 support. Useful to replicate crypto-backend builds without ED25519, such as wolfSSL. - openssl: fix unused variable and function warnings with all supported `LIBSSH2_NO_*` options enabled. - mbedtls: fix misplaced `#endif` leaving out the required internal public function `libssh2_supported_key_sign_algorithms()`. - mbedtls: add missing prototype for two internal public functions. - delete a redundant block. All `NO` options: ```shell CPPFLAGS=' -DLIBSSH2_NO_MD5 -DLIBSSH2_NO_HMAC_RIPEMD -DLIBSSH2_NO_DSA -DLIBSSH2_NO_RSA -DLIBSSH2_NO_RSA_SHA1 -DLIBSSH2_NO_ECDSA -DLIBSSH2_NO_ED25519 -DLIBSSH2_NO_AES_CTR -DLIBSSH2_NO_BLOWFISH -DLIBSSH2_NO_RC4 -DLIBSSH2_NO_CAST -DLIBSSH2_NO_3DES' ``` Closes #1044 - cmake: avoid `list(PREPEND)` for compatibility `list(PREPEND)` requires CMake v3.15, our minimum is v3.1. `APPEND` should work fine for headers anyway. Also fix a wrongly placed comment. Ref: https://cmake.org/cmake/help/latest/command/list.html#prepend Regression from 1e3319a167d2f32d295603167486e9e88af9bb4e Closes #1043 - checksrc: verify label indent, fix fallouts Also update two labels to match the rest of the source. checksrc update credit: Emanuele Torre @emanuele6 Ref: https://github.com/curl/curl/pull/11134 Closes #1042 - tidy-up: minor nits - ci: drop default shared/static configuration options Both autotools and cmake build both shared and static lib by default. Ref: 896154bc17f000c0a1bb89b74bc879692ac0d47c Delete configuration enabling these explicitly in CI jobs. Cherry-picked from #1036 Closes #1036 - cmake: enable shared libssh2 library by default This brings default behaviour in sync with autotools, which builds both lib flavours by default. (Notice that on Windows, autotools includes the Windows Resource in the static library, when building both at the same time. CMake doesn't have this issue.) Enabling both lib flavours has a side-effect when using non-MinGW toolchains (e.g. MSVC): to resolve the filename conflict between import and static libraries, we add a suffix to the static lib, naming it `libssh2_static.lib`. This can break dependent builds relying on `libssh2.lib` for linking the static libssh2. Workarounds: - disable either shared or static libssh2 via `-DBUILD_STATIC_LIBS=OFF` or `-DBUILD_SHARED_LIBS=OFF`. This results in a libssh2 library (either static or shared) without a prefix: `libssh2.lib`. - set a custom static library suffix via: `-DSTATIC_LIB_SUFFIX=_my_static`. Resulting in `libssh2_my_static.lib`, and import library `libssh2.lib`. - set a custom import library suffix via: `-DIMPORT_LIB_SUFFIX=_my_implib`. Resulting in `libssh2_my_implib.lib` import library, and static library `libssh2.lib`. - customize the default static/import library suffix (incl. extension) via `-DCMAKE_STATIC_LIBRARY_SUFFIX=_my_static_suffix.lib` or `-DCMAKE_IMPORT_LIBRARY_SUFFIX=_my_import_suffix.lib`. Cherry-picked from #1036 - cmake: tweak static/import lib name collision avoidance logic The collision issue affects (typically) MSVC, when building both shared and static libssh2 in one go. Ref: https://stackoverflow.com/questions/2140129/what-is-proper-naming-convention-for-msvc-dlls-static-libraries-and-import-libr Initially we handled this by appending the `_imp` suffix to the import library filename. This is how curl tackles this, but on a second look, this solution seem to be accidental and has no widespread use. It seems more widely accepted to use the '_static' suffix for the static library. This patch implements this. (MinGW, Cygwin and unixy platforms are not affected by this issue.) Follow-up to 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1 Cherry-picked from #1036 - cmake: add `IMPORT_LIB_SUFFIX` (like `STATIC_LIB_SUFFIX`) Allow resolving the import/static library name collision also by setting a custom _import_ library name suffix. Follow-up to 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1 Cherry-picked from #1036 - ci: do not disable shared lib with msys2/autotools in GHA Cherry-picked from #1036 - Makefile.mk: fix `DYN=1 test` by skipping tests needing static lib `DYN=1` means to build examples/tests against the shared libssh2. Before this patch this was broken for building tests. This patch skips building tests that require the static libssh2 library, so the build now succeeds. Also move the list of tests that require static lib from `CMakeLists.txt` to `Makefile.inc`, so that we can reuse it in `Makefile.mk`. Couldn't find a way to also reuse it in `Makefile.am`. Move the `Makefile.am` specific definitions close to the shared list, to make it easier to keep them synced. Cherry-picked from #1036 - ci: make one of the AppVeyor CMake jobs shared-only This build combination did not have a CI test before. Cherry-picked from #1036 - cmake: allow tests with `BUILD_STATIC_LIBS=OFF` Before this patch, the CMake build did not allow to disable static libssh2 library while also building tests. This patch removes this constraint, and makes this combination possible. In this case the 3 (at the moment) tests that require a static libssh2 library, are skipped from the build and test runs. Cherry-picked from #1036 - build: fix to set `-DLIBSSH2DEBUG` for tests Required for tests using libssh2 internals. These are the ones requiring the libssh2 _static_ lib. Before this patch, `src` and `tests` declared the `session` structure differently, due to extra struct members added with the `LIBSSH2DEBUG` macro set. But, the macro was only set for `src` when using CMake. At runtime this caused struct members to be at different offsets between lib and test code, resulting in the test failures below. Due to another bug in the affected test, these failures did not reflect in the exit code, which always returned success, so this went unnoticed for a good while. Fixed in: 84d31d0ca7b647ad4c2aa92bf8f4a94b233f5d3b ``` Start 5: test_auth_keyboard_info_request [...] 5: Test case 1 passed 5: Test case 2 passed 5: Test case 3: expected return code to be 0 got -1 5: Test case 4: expected last error code to be "-6" got "-38" 5: Test case 5: expected last error code to be "-6" got "-38" 5: Test case 6: expected last error code to be "-6" got "-38" 5: Test case 7: expected last error message to be "Unable to decode keyboard-interactive number of keyboard prompts" got "userauth keyboard data buffer too small to get l 5: Test case 8: expected last error code to be "-41" got "-38" 5: Test case 9: expected return code to be 0 got -1 5: Test case 10: expected return code to be 0 got -1 5: Test case 11: expected last error code to be "-6" got "-38" 5: Test case 12: expected last error message to be "Unable to decode user auth keyboard prompt echo" got "userauth keyboard data buffer too small to get length" 5: Test case 13: expected return code to be 0 got -1 5: Test case 14: expected return code to be 0 got -1 5: Test case 15: expected last error code to be "-6" got "-38" 5: Test case 16: expected last error code to be "-6" got "-38" 5: Test case 17: expected last error code to be "-6" got "-38" 5: Test case 18: expected last error code to be "-6" got "-38" ``` Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46925869/job/i9uasceu3coss0i2#L440 Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46983040/job/c3vag25c26a77lyr#L485 Cherry-picked from #1037 Closes #1037 - test_auth_keyboard_info_request: fix to return failure Before this patch, this test returned success even when one of its tests failed. Fix it by returning 1 in case any of the tests fails. This issue masked a CMake build bug with logging enabled. Subject to an upcoming patch. Cherry-picked from #1037 - test_auth_keyboard_info_request: fix indentation Cherry-picked from #1037 - tidy-up: move comment off from copyright header Cherry-picked from #1037 - ci: enable shared libs in msys2/macOS cmake builds Shared libs improve example/tests build times. For "unity" builds the overhead of building shared lib is negligible, so this even reduced the overall build-time. Follow-up to 3d64a3f5100f7f4cf52202396eb4f1c3f3567771 Follow-up to d93ccf4901ef26443707d341553994715414e207 Tests: https://github.com/libssh2/libssh2/actions/runs/4906586658: unity builds enabled https://github.com/libssh2/libssh2/actions/runs/4906925743: unity builds enabled + parallel msys2 builds https://github.com/libssh2/libssh2/actions/runs/4906777629: unity + shared lib (this commit) https://github.com/libssh2/libssh2/actions/runs/4906927190: unity + shared lib (this commit) + parallel msys2 builds Consider making shared libs enabled by default also in CMake, to sync it with autotools? Closes #1035 - ci: add missed --parallel 3 from msys2 cmake builds Follow-up to 3d64a3f5100f7f4cf52202396eb4f1c3f3567771 - cmake: add and test "unity" builds "Unity" (aka "jumbo", aka "amalgamation" builds concatenate source files before compiling. It has these benefits for example: faster builds, improved code optimization, cleaner code. Let's support and test this. - enable unity builds for some existing CI builds to test this build scenario. - tune `UNITY_BUILD_BATCH_SIZE` size. - disable unity build for example and test programs (they use one source each already). You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. Supported by CMake 3.16 and newer. Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html Closes #1034 - tests: simplify passing `srcdir` to tests Before this patch libssh2 used a variety of solutions to pass the source directory to tests: `FIXTURE_WORKDIR` build-time macro (cmake), `FIXTURE_WORKDIR` envvar (unused), setting `srcdir` manually (autotools), setting current directory (cmake), and also `builddir` envvar (autotools) for passing current working dir to `mansyntax.sh`. This patch reduces this to using existing `srcdir` with autotools and setting it ourselves in CMake. This was mostly enabled by this recent patch: 4c9ed51f962f542b98789b15bedaaa427f4029a2 Details: - cmake: replace baked-in `FIXTURE_WORKDIR` macro with env. Added in 54bef4c5dad868a9d45fdbfca9729b191c0abab5 #198 (2018-03-21) - rename `FIXTURE_WORKDIR` to `srcdir`, to match autotools. - cmake: add missing `srcdir` for algo and sshd tests. - session_fixture: stop `chdir()`-ing, rely on prefixing with `srcdir`. Changing current directory should be unnecessary after 4c9ed51f962f542b98789b15bedaaa427f4029a2 #801 (2023-02-24), that prefixes referenced input filenames with the `srcdir` envvar. The `srcdir` envvar was already exported by autotools, and now we're also setting it from CMake. - cmake: stop setting `WORKING_DIRECTORY`, rely on `srcdir` env. `WORKING_DIRECTORY` is no longer necessary, after passing `srcdir` to all tests, so they can find our source tree and keys/etc in it regardless of the current directory. Also this past commit hints that `WORKING_DIRECTORY` wasn't always working for this purpose as expected: "tests: Xcode doesn't obey CMake's test working directory" Ref: https://github.com/libssh2/libssh2/pull/198/commits/10a5cbf945abcc60153ee3d59284d09fc64ea152 - autotools: delete explicit `srcdir` for test env. Added in 13f8addd1bc17e6c55d52491cc6304319ac38c6d (2015-07-02) automake documents `srcdir` as exported to the test environment: https://github.com/autotools-mirror/automake/blob/c04c4e8856e3c933239959ce18e16599fcc04a8b/doc/automake.texi#L9302-L9304 https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html It's mentioned in the docs back in 1997 and got a regression test in 2012. We can safely assume it to be available without setting it ourselves. - autotools: delete explicit `builddir`. Added in 13f8addd1bc17e6c55d52491cc6304319ac38c6d (2015-07-02) It seems this wasn't necessary to make the above fix work, and `mansyntax.sh` is able to figure out the build workdir by reading `$PWD`. Our out-of-tree and `make distcheck` CI builds also work without it. Let us know if there is a scenario we're missing and needs this. Closes #1032 - src: fix `libssh2_store_*()` for >u32 inputs `_libssh2_store_str()` and `_libssh2_store_bignum2_bytes()` accept inputs of `size_t` max, store the size as 32-bit unsigned integer, then store the complete input buffer. With inputs larger than `UINT_MAX` this means the stored size is smaller than the data that follows it. This patch truncates the stored data to the stored size, and now returns a boolean with false if the stored length differs from the requested one. Also add `assert()`s for this condition. This is still not a correct fix, as we now dump consistent, but still truncated data which is not what the caller wants. In future steps we'll need to update all callers that might pass large data to this function to check the return value and handle an error, or make sure to not call this function with more than UINT_MAX bytes of data. Ref: c3bcdd88a44c4636818407aeb894fabc90bb0ecd (2010-04-17) Ref: ed439a29bb0b4d1c3f681f87ccfcd3e5a66c3ba0 (2022-09-29) Closes #1025 - cmake: limit WinCNG to Windows After deleting the `bcrypt.h` check, no check remained. Restore a `WIN32` check here to ensure WinCNG is not enabled outside Windows. Follow-up to 1289033598546ee5089ff0fc4369d24e1e2be81f Tested-in #1032 - cmake: move `CMAKE_VS_GLOBALS` setting to CI configs To not force this setting for local builds where they might serve a good purpose. It makes our CI runs slightly faster and we don't need to track file changes in unattended, single, CI runs. Cherry-picked from #1031 - cmake: prefill for faster config phase on Windows Prefill known detection results on Windows with MinGW and MSVC, to avoid spending time on detecting these on every cmake configuration run. With MinGW + clang and MSVC, this elminates all detections. With MinGW + gcc, it reduces them to 3. Cherry-picked from #1031 - libssh2_setup.h: set `HAVE_INTTYPES_H` for MSVC To sync up the hand-crafted config with actual detection results by CMake and autotools. Sources compiled fine without it anyway. Cherry-picked from #1031 - cmake: re-add `select()` detection (regression) `select()` detection suffered two regressions: First I accidentally deleted it for non-Windows [1]. Then the Windows-specific setting got missed from the generated `libssh2_config.h` after a rearrangement in `CMakeLists.txt` files. [1] 31fb8860dbaae3e0b7d38f2a647ee527b4b2a95f (2023-03-07) [2] 803f19f004eb6a5b525c48fff6f46a493d25775c (2023-04-18) This patch restores detection. For Windows, enable it unconditionally, not only for speed reasons, but because detection needs `ws2_32`, and even that is broken on the x86 platform. According to the original `cmake/SocketLibraries.cmake`, caused by a calling convention mismatch. FWIW autotools detects it correctly. Cherry-picked from #1031 - ci: merge make job into msys2 section, enable zlib + openssl Follow up to dd625766271a0ba13f5ac661bdc2fa40bbfa580a Cherry-picked from #1030 - ci: add missing timeouts for autotools tests Cherry-picked from #1030 - ci: add mingw-w64 clang and gcc CMake jobs Cherry-picked from #1030 - cmake: assume `bcrypt.h` with WinCNG autotools already didn't check for `bcrypt.h`, and such check is only required for old/legacy mingw without obsolete/incomplete Windows headers. curl deprecated old-mingw support just recently and will delete support in September 2023. This patch saves some complexity and detection time by dropping this check for CMake. Meaning that mingw-w64 is now required to compile libssh2 when using the WinCNG backend for 32-bit builds. Other backends and CPU platforms are not affected. Ref: https://github.com/curl/curl/commit/e4d5685cb5d6eb07e1b43156fd7e3ba3563afba5 Closes #1026 - cmake: do not check for `poll()` on Windows While it seems to exist on mingw in theory, it's not detected as of this writing. It also has issues, and not ready for production use: https://stackoverflow.com/questions/1671827/poll-c-function-on-windows On MSVC it's even less supported. Skip checking this to save CMake detection time. Closes #1027 - agent_win: make a struct static and other build improvements Also: - merge back `agent.h` into `agent.c` where it was earlier. Ref: c998f79384116e9f6633cb69c2731c60d3a442bb - introduce `HAVE_WIN32_AGENT` internal macro. - fix two guards to exclude more code unused in UWP builds. Follow-up to 1c1317cb768688eee0e5496c72683190aaf63b29 Closes #1028 - tidy-up: formatting nits Whitespace and redundant parenthesis in `return`s. Closes #1029 GitHub (3 May 2023) - [Nick Woodruff brought this change] sftp: parse attribute extensions, if present, to avoid stream parsing errors (#1019) Prevents directory listing errors when attribute extensions are present by advancing stream parsing past extensions. Viktor Szakats (3 May 2023) - tests: merge `sshd_fixture.sh` into `test_sshd.test` Merge the loop executing multiple tests and the script that actually launches the tests into a single script. This same script is now called from both autotools and CMake. autotools loads the list of tests from `Makefile.inc`, CMake passes it via the command-line. It's also possible to call the script manually with a custom list of tests or individual ones. With this setup we're now launching a single sshd session for all tests, instead of launching and killing it for each test. This did not improve reliability of these test on CI machines, and it's easy to go back to the previous behaviour if necessary. Also: - allow passing custom sshd options via `SSHD_FLAGS`. - add `SSHD_TESTS_LIMIT_TO` to limit the number of tests to its value. E.g. `SSHD_TESTS_LIMIT_TO=1` executes the first test only. Meant for debugging. - use `ssh` to test the connection (if available) instead of fixed amount of wait. Made to also work on Windows. - set `PermitRootLogin yes` in `sshd`, to allow running tests as root. - show `sshd` path and version. Cherry-picked from #1017 (the last one) Closes #1024 - ci: make sure to run tests after all builds in GHA Whenever possible. Due to flakiness/hangs/timeouts, keep sshd tests disabled on Windows and macOS. Also keep Docker tests disabled on these platforms, they do not work: GHA Windows: ``` no matching manifest for windows/amd64 in the manifest list entries ``` GHA macOS: ``` sh: docker: command not found ``` It's not possible to run UWP and ARM64 binaries: UWP: ``` Test #2: test_simple ......................Exit code 0xc0000135 ``` Needs but doesn't find: `VCRUNTIME140_APP.dll`. ARM64 ``` D:/a/libssh2/libssh2/bld/tests/Release/test_ssh2.exe: cannot execute binary file: Exec format error ``` Cherry-picked from #1017 - tests: disable sshd tests on Windows via new options Instead of using hacks inside the build systems. `SSHD` variable added to GitHub Actions is not currently used. Added there to make it easy to experiment with these tests and the path is non-trivial to discover. Using the Windows built-in sshd server is another option (haven't discovered its path yet). Cherry-picked from #1017 - tests: add cmake/autotools options to disable running tests autotools: - `--disable-docker-tests` - `--disable-sshd-tests` cmake: - `RUN_DOCKER_TESTS` - `RUN_SSHD_TESTS` Update automake and ci to use this new flag and delete former logic of relying on Windows detection and `HOST_WINDOWS`. Also fix honoring this when running `test_read_algos.test`. This allows to disable these individually and on per-CI/local-job basis. To run as much tests as the env allows. Cherry-picked from #1017 - ci: add `make distcheck` job Cherry-picked from #1017 - ci: switch to out-of-tree autotools builds Cherry-picked from #1017 - ci: restore parallel builds with cmake Also add missing -j3 for macOS builds. Partial revert of 0d08974633cfc02641e6593db8d569ddb3644255 Cherry-picked from #1017 - ci: sync names, steps, syntax, build dirname between jobs Also: - delete an unused 64-bit option for Linux (all jobs are 64-bit). - fix to not install libgcrypt and openssl when doing mbedTLS builds. [ Empty lines after multiline run commands are solely to unbreak my editor's syntax highlighting. They can be deleted in the future ] Cherry-picked from #1017 - ci: add `Makefile.mk` test, with `LIBSSH2_NO_*` options Cherry-picked from #1017 - Makefile.mk: use Makefile.inc from example and tests Instead of assembling the list using `$(wildcard ...)`. Also split off a `tests/Makefile.inc` from `tests/Makefile.am`. With its simpler syntax, this also allows to delete some complexity from the CMake loader. Cherry-picked from #1017 - example, tests: fix ssh2 to correctly return failure Before this patch ssh2 and test_ssh2 returned success even if the session failed at `libssh2_session_handshake()` or after. This patch depends on cda41f7cb87c3af5258ba48ccef19d3efdbd3d3b, that fixed running test_ssh2 on Windows via sshd_fixture. Cherry-picked from #1017 - tests: set -e -u in shell scripts Cherry-picked from #1017 - cmake: use shared libs again in example and tests Re-sync with autotools and v1.10.0 behavior. This improves build times. It also allows to stop building our special shared test target to test shared builds. Follow-up to 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1 Cherry-picked from #1017 Closes #1022 - tests: retry KEX failures when using the WinCNG backend Twice. This tests are flaky and we haven't figured out why. In the meantime use this workaround to test and log these issues, but also ensure that CI run aren't flagged red because of it. Also: - kex: add debug message when hostkey `sig_verify` fails, to help tracking WinCNG KEX failures. - test_ssh2: also add retry logic. I'm not quite sure this is correct. Please let me know. - session_fixture: bump up `src_path` slots to fit retries and show message when hitting the limit. - session_fixture: clear `kbd_password` static variable after use. - session_fixture: close and deinit socket after use. - session_fixture: deinit libssh2 after use. Ref: #804 #846 #979 #1012 #1015 Cherry-picked from #1017 Closes #1023 - example, test_ssh2: shutdown socket before close Syncing them with `tests/session_fixture.c`. Cherry-picked from #1017 - ci.yml: fix indentation [ci skip] Cherry-picked from #1017 - Makefile.mk: make tests depend on runner lib Cherry-picked from #1017 - build: compile agent_win.c via agent.c Silences these warnings on non-Windows: ``` ranlib: file: libssh2.a(agent_win.c.o) has no symbols ``` Cherry-picked from #1017 - cmake: delete obsolete comment Follow-up to 80175921638fa0a345237d23206a2ad1644cdd9b Cherry-picked from #1017 - checksrc.sh: fix it to run from any current directory Also silence a shellcheck warning. Cherry-picked from #1017 - ISSUE_TEMPLATE: ask for crypto backend version Also fix casing in backend names. Cherry-picked from #1017 - tests: fix newlines in test keys for sshd on Windows Make sure these files get LF newlines on checkout. Before this patch a checked out libssh2 Git repository may have used CRLF newlines in text files, include test keys. Private keys with CRLF newlines could confuse sshd on Windows: ``` # sshd version: 'OpenSSH_9.2, OpenSSL 1.1.1t 7 Feb 2023' Unable to load host key "/d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key": invalid format Unable to load host key: /d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key ``` Ref: https://github.com/libssh2/libssh2/actions/runs/4846188677/jobs/8635575847#step:6:39 Cherry-picked from #1017 - cmake: move option descriptions next to definition Cherry-picked from #1017 - checksrc: sync with curl There were no new issues detected. Cherry-picked from #1017 - openssl: enable AES-GCM with wolfSSL Follow-up to 3c953c05d67eb1ebcfd3316f279f12c4b1d600b4 #797 There is pending issue with wolfSSL, where encryption/decryption is not working (both with and without this patch). Ref: #1020 Cherry-picked from #1017 - appveyor: add a UWP OpenSSL 3 build Cherry-picked from #1017 - appveyor: skip `before_test` when not doing tests Also merge `before_test` section into `test_script`. Cherry-picked from #1017 - docs: delete two stray characters Cherry-picked from #1017 - tidy-up: avoid expression 'of course' Cherry-picked from #1017 - tidy-up: avoid word 'just' Cherry-picked from #1017 - tidy-up: avoid word 'simply' Cherry-picked from #1017 - tests: teach to use the `USERNAME` envvar on Windows Necessary to pick the correct local username when run on Windows. Cherry-picked from #1017 - test_ssh2: support `FIXTURE_TRACE_ALL*` envvars Cherry-picked from #1017 - tidy-up: add missing newline to error msg, formatting Also: - fix indent - lowercase variables names - fix formatting in `src/global.c` Cherry-picked from #1017 - appveyor: wait more for SSH connection from GHA Cherry-picked from #1017 - ci: restrict permissions in GitHub Actions Cherry-picked from #1017 - build: fix autoreconf warnings - update `AC_HELP_STRING' to 'AS_HELP_STRING`: ``` configure.ac:[...]: warning: The macro `AC_HELP_STRING' is obsolete. ``` "AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works already since 2.59 so bump the minimum required version to that." Ref: https://github.com/curl/curl/commit/a59f04611629f0db9ad8e768b9def73b9b4d9423 - simplify to avoid: ``` src/Makefile.inc:48: warning: variable 'EXTRA_DIST_SOURCES' is defined but no program or src/Makefile.inc:48: library has 'DIST' as canonical name (possible typo) ``` Regression from 2c18b6fc8df060c770fa7e5da704c32cf40a5757 - `AC_TRY_LINK`/`AC_TRY_COMPILE`: ``` configure.ac:335: warning: The macro `AC_TRY_COMPILE' is obsolete. configure.ac:335: warning: The macro `AC_TRY_LINK' is obsolete. ``` - `libtool`-related ones: ``` configure.ac:70: warning: The macro `AC_LIBTOOL_WIN32_DLL' is obsolete. configure.ac:70: warning: AC_LIBTOOL_WIN32_DLL: Remove this warning and the call to _LT_SET_OPTION when you configure.ac:70: put the 'win32-dll' option into LT_INIT's first parameter. configure.ac:71: warning: The macro `AC_PROG_LIBTOOL' is obsolete. ``` Using code copied from curl: https://github.com/curl/curl/blob/9ce7eee07042605045dcfd02a6f5b38ad5c8a05d/m4/xc-lt-iface.m4#L157-L163 - delete commented and obsolete `AC_HEADER_STDC`. - formatting. Most cherry-picked from `autoupdate` updates. Cherry-picked from #1017 Closes #1021 - docker-bridge.ps1: use native newlines Also add a shebang and exec flag to ease testing/handling on *nix. PowerShell accepts both LF and CRLF. Cherry-picked from #1017 GitHub (1 May 2023) - [Zenju brought this change] sftp: remove packet limit for directory reading (#791) Currently libssh2 cannot read huge directory listings when the package size of `LIBSSH2_SFTP_PACKET_MAXLEN` (256KB) is hit. For example AWS always sends a single package with all files of a directory, no matter how big it is: https://freefilesync.org/forum/viewtopic.php?t=10020 Package size is probably around 7MB in this case! `LIBSSH2_SFTP_PACKET_MAXLEN` is a good idea in general, but there doesn't seem to be a one size fits all. While almost all(?) SFTP responses come in very small packages, I believe the `SSH_FXP_READDIR` request should be exempted. The proposed patch, enhances the package size reading to include parsing the full SFTP packet header. And in case a package is of type `SSH_FXP_NAME` and matches an expected `readdir_request_id`, it does not fail if `LIBSSH2_SFTP_PACKET_MAXLEN` is hit. The chances of accidentally hiding data-corruption are pretty non-existent, because both SFTP `request_id` and packet type must match. No change in behavior otherwise. Best, Zenju Previous discussion: #268 #269 With the above changes, the `LIBSSH2_SFTP_PACKET_MAXLEN` value could (and should?) probably be set back to a small number again. Integration-patches-by: Viktor Szakats Viktor Szakats (28 Apr 2023) - checksrc: update and apply fixes Update to latest revision and fix new issues detected. Closes #1014 - ci: add macOS CI jobs + fix issues revealed Add macOS CI jobs, both cmake and autotools for all supported crypto backends (except BoringSSL), with debug, zlib enabled. Without running tests. It also introduces OpenSSL 1.1 into the CI with a non-MSVC compiler. Credits to curl's `macos.yml`, that I used as a base. Fix these issues uncovered by the new tests: - openssl: fix warning when built with wolfSSL, or OpenSSL 1.1 and earlier. CI missed it because apparently the only OpenSSL 1.1 test we had used MSVC, which did not complain. ``` ../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable] unsigned long sslError; ^ ``` Regression from 097c8f0dae558643d43051947a1c35b65e1c5761 - pem: add hack to build without MD5 crypto-backend support. The Homebrew wolfSSL build comes with MD5 support disabled. We can expect this becoming the norm. FIPS also requires MD5 disabled. We deleted the same hack from `hostkey.c` a month ago: ad6aae302aaec84afbfacf0c1dfdc446d46eaf21 A better fix would be to guard the MD5 logic with our `LIBSSH2_MD5` macro. ``` pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'? unsigned char secret[2*MD5_DIGEST_LENGTH]; ^~~~~~~~~~~~~~~~~ SHA_DIGEST_LENGTH ``` Regression from 386e012292a96fcf0dc6861588397845df0aba2c - `configure.ac`: add crypto libs late. Fix it by adding crypto libs to `LIBS` at the end of the configuration process. Otherwise `configure` links crypto libs while doing feature tests, which can cause unwanted detections. For example LibreSSL publishes the function `explicit_bzero()`, which masks the system alternative, e.g. `memset_s()` on macOS. Then when trying to compile libssh2, its declaration is missing: ``` bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration] _libssh2_explicit_zero(ciphertext, sizeof(ciphertext)); ^ ../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero' ^ ``` Regression from 4f0f4bff5a92dce6a6cd7a5600a8ee5660402c3f - cmake: fix to list our own include directory before the crypto libs', when building tests. Otherwise a global crypto header path, such as `/usr/local/include`, containing an external `libssh2.h` of a different version, could cause weird errors: ``` cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \ -I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \ -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk \ -mmacosx-version-min=12.6 -MD -MT \ tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \ -MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d \ -o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \ [...]/libssh2/tests/test_aa_warmup.c ``` ``` [ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4: In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42: In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43: /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier LIBSSH2_AUTHAGENT_FUNC((*authagent)); ^ /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int] LIBSSH2_AUTHAGENT_FUNC((*authagent)); ^ /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities)); ^ /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int] LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities)); ^ /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback)); ^ /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int] LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback)); ^ 6 errors generated. ``` - `tests/session_fixture.h`: delete duplicate `libssh2.h`, `libssh2_priv.h` already includes it. Follow-up to a683133dfe96de126194f58f183131a84c7d36a2 CI logs with these errors: https://github.com/libssh2/libssh2/actions/runs/4824079094 https://github.com/libssh2/libssh2/actions/runs/4824270819 curl's `macos.yml`: https://github.com/curl/curl/blob/da2470de96e94e1c8d276b9ae6e4c97c2cf54239/.github/workflows/macos.yml Tidying-up while here: - tests/session_fixture.h: delete duplicate `libssh2.h`. `libssh2_priv.h` includes it already. Follow-up to a683133dfe96de126194f58f183131a84c7d36a2 - ci.yml: yamllint warnings and formatting. - ci.yml: msvc section formatting and step-naming sync with macOS. Follow-up to f4a4c05dc3bcd62ecaa1b0cac5997faefe16c83f - ci.yml: enable `--enable-werror` for msys2 jobs. Follow-up to 71cae949d577fdd632a271da0bec89f977dc5dd2 - appveyor.yml: show OpenSSL versions, link to image content. Closes #1013 - ci: convert `docker-bridge.bat` to shell script Convert `ci/appveyor/docker-bridge.bat` to a POSIX shell script. Also bump the tunnel to use ed25519 (was RSA-2048). Closes #997 - kex: use distinctive error strings Use unique error strings to help localize errors. Closes #1011 - tidy-up: C header use - drop unused or duplicate C headers. - add missing ones (that worked by chance). (`string.h`, `stdlib.h`) - mention the functions that need certain headers. - move some headers from crypto header to crypto C source. - reorder headers in some places. - simplify the #if tree for `sys/select.h` in `libssh2_priv.h`. - move scp-specific macros next to their header to `scp.c` Follow-up to 5db836b2a829c6fff1e8c7acaa4b21b246ae1757 Closes #999 - tidy-up: text nits, English contractions [ci skip] In input/output text and docs mostly. - ci: add MSVC and UWP builds to GitHub Actions - add MSVC jobs to GitHub Actions. They are similar to the 'Build-only' jobs we have on AppVeyor CI, though only the ARM64 Windows one is identical. Major disadvantage is that we don't run tests here. Major advantage is they only take a few minutes to complete, compared to an hour on AppVeyor, so WinCNG build results now appear quicker. Docker tests might be possible, but my light attempts failed. Finding ZLIB also failed, so we still miss an MSVC test with it. Tool versions as of now: Server 2022, VS2022, OpenSSL 1.1.1 - add UWP builds for both ARM64 and x64. This hasn't been CI tested before. (We could probably enable UWP on AppVeyor CI as well. I haven't tried.) - fix two uncovered UWP issues in tests. - rename internal macro `LIBSSH2_WINDOWS_APP` to `LIBSSH2_WINDOWS_UWP`. Follow-up to 2addafb77b662e64248d156c71c69b91ba7b926e - fold long lines and quote truthy values in `.github/workflows/ci.yml`. Closes #1010 - session_fixture: avoid no-op `chdir(getcwd())` If no `FIXTURE_WORKDIR` macro or envvar is present to set the cwd, avoid querying the cwd and then calling chdir with the result. Ref: 54bef4c5dad868a9d45fdbfca9729b191c0abab5 (patch) Ref: 10a5cbf945abcc60153ee3d59284d09fc64ea152 (individual commit) Closes #1009 - tests/sshd_fixture.sh: convert back to POSIX There was no strong reason to require bash. Let's use POSIX shell like before the recent overhaul. Follow-up to a459a25302a31f6e2aba3c4e15b1472b83b596fc Closes #1008 GitHub (26 Apr 2023) - [Miguel de Icaza brought this change] If SFTP fails to initialize, do not busy loop waiting for IO to happen (#720) Currently SFTP's init will busy loop waiting for the channel to close, even if the underlying transport returns EAGAIN. While this works for sockets, it might not work out if you have a different transport that needs to do some additional processing on the side. Integration-patches-by: Viktor Szakats Viktor Szakats (26 Apr 2023) - docs: simplify `.TH` header & other cleanups [ci skip] - simplify `.TH` headers. - delete empty lines before sections. - update template with an `AVAILABILITY` section. Left libssh2 version number in the `.TH` header for entries without an `AVAILABILITY` section, or where there was a different version number there. - tidy-up: formatting nits [ci skip] - vms: fix to include `sys/socket.h` Due to a typo in the `HAVE_*` macro, this header was never included. A comment suggests that `socklen_t` is not defined on VMS and defines it manually. This symbol is usually in `sys/socket.h`, so the typo may have been the reason for it to be missing. Closes #1007 - build: fix `make distcheck` regressions - add #included C files to `EXTRA_DIST`. Regression from 4f0f4bff5a92dce6a6cd7a5600a8ee5660402c3f - fix `tests/sshd_fixture.sh` to not write into the test dir, by using a pre-assembled `TrustedUserCAKeys` file. Update `Dockerfile` too to use this. Regression from a459a25302a31f6e2aba3c4e15b1472b83b596fc Also update `tests/sshd_fixture.sh` to use `openssh_server/authorized_keys` like `Dockerfile` does. And a few more cosmetic updates. Closes #1006 - libssh2_priv.h: assume `HAVE_LONGLONG` Unless I'm missing something, it looks like `libssh2.h` has been using `libssh2_int64_t` unconditionally since at least 2010-04-17 when `libssh2_scp_send64()` landed via commit be9ee7095e2d5021985f57d88f5f889d3c2b9d8f. This makes it redundant to detect `HAVE_LONGLONG` to fallback to a 32-bit `scpRecv_size` in `libssh2_priv.h`. Then deal with possible combinations of this flag and `strtoll()` options, which was error-prone. Instead, assume in `libssh2_priv.h` that we have `libssh2_int64_t`, and use it always. For MSVC, this means `_MSC_VER` `1310` (from year 2003) is now required. Based on the above, this was already so before this patch. If there happens to be no 64-bit `strtoll()` detected, fall back to the 32-bit `strtol()` (this should never happen with MSVC, and probably neither with any other reasonably modern toolchain.) Also make sure to set `HAVE_STRTOI64` for older, non-CMake, MSVC builds (e.g. `Makefile.mk` or `NMakefile` ones). Closes #1002 GitHub (26 Apr 2023) - [Miguel de Icaza brought this change] fix a couple of small regressions (#1004) - openssl: fix potentially missing `ERR_*` constants by including `openssl/err.h`. This could happen with recent version of Xcode or when building against OpenSSL built with the `OPENSSL_NO_ENGINE` option. Regression from 097c8f0dae558643d43051947a1c35b65e1c5761 (#789) - channel: fix an issue that would corrupt the data stream when attempting to initialize the agent in non-blocking mode, as it is necessary to propagate the `EAGAIN` signal upstream when the transport returns `EAGAIN`. Regression from bc4e619e76071393e466c29220fc4ef5764c2820 (#752) - packet: the current code does not set the state machine upon reaching this point which means that if the code is suspended due to the transport returning an `EAGAIN`, this will re-initialize the structure every time. The issue is that this keeps assigning a new channel-id downstream, which does not match the initial channel-id that is initially generated, causing a lookup later to fail as there is no matching channel. Regression from bc4e619e76071393e466c29220fc4ef5764c2820 (#752) Viktor Szakats (26 Apr 2023) - tidy-up: `gettimeofday()` fallback and use Simplify the way we handle `gettimeofday()` fallback for platforms without native support or without any support. Make it similar to how we handle `snprintf()`. In case of no native `gettimeofday()` support and a non-Windows platform, our local fallback returns zero in `tv_usec` and `tv_sec`, ending up with a zero `timeout_remaining` in `session.c`, same as before this patch. Also: - drop unused `sys/time.h` headers. - fix our fallback code to compile with any Windows compilers (not just MSVC) - delete unnecessary casts. Closes #1001 - libssh2_priv.h: fix checksrc warning [ci skip] Regression from 9ef75298fae0728305d9d38ba1e3c838ad0513f7 - libssh2_priv.h: whitespace fixes cont. [ci skip] - libssh2_priv.h: whitespace fixes [ci skip] - cmake: use portable mkdir for tests/coverage target [ci skip] Makes `make coverage` work without a POSIX mkdir. Tested locally. Ref: https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-E-arg-make_directory - kex: fix overlapping memcpy() to memmove() Noticed this when libasan started kicking out errors when sending in MACs preferences that were not supported yet. Reported-by: fourierules on github Fixes #611 Closes #1000 - test/CMakeLists.txt: reuse `Makefile.am` librunner source list Follow-up to a459a25302a31f6e2aba3c4e15b1472b83b596fc Closes #998 GitHub (25 Apr 2023) - [Zenju brought this change] openssl: fix misleading error message if wrong passphrase (#789) Fixes #608 Viktor Szakats (25 Apr 2023) - tidy-up: tiny nits [ci skip] - tests: improve running tests TL;DR: Sync test builds between autotools and CMake. Sync sshd configuration between Docker and non-Docker fixtures. Bump up sshd_config for recent OpenSSH releases. This also opens up the path to have non-Docker tests that use a local sshd process. Though sshd is practically unusable on Windows CI machines out of the box, so this will need further efforts. Details: - cmake: run sshd fixture test just like autotool did already. - sync tests and their order between autotools and CMake. It makes `test_aa_warmup` the first test with both. - cmake: load test lists from `Makefile.am`. Needed to update the loader to throw away certain lines to keep the converted output conform CMake syntax. Using regexp might be an alternative way of doing this, but couldn't make it work. - cmake: use the official way to configure test environment variables. Switch to syntax that's extendable. - cmake: allow to run the same test both under Docker and sshd fixture. Useful for testing the sshd fixture runner, or how the same test behaves in each fixture. - update test fixture to read the username from `USER` envvar instead of using the Dockfile-specific hardwired one, when running outside Docker. - rework `ssh2.sh` into `sshd_fixture.sh`, to: - allow running any tests (not just `test_ssh2`). - configure Docker tests for running outside Docker. - fixup `SSHD` path when running on Windows (e.g. in AppVeyor CI). Fixes: `sshd re-exec requires execution with an absolute path` - allow overriding `PUBKEY` and `PRIVKEY` envvars. - allow overriding `ssh_config` via `SSHD_FIXTURE_CONFIG`. - prepare support for running multiple tests via sshd_fixture. Add a TAP runner for autotools and extend CMake logic. The TAP runner loads the test list from `Makefile.am`. Notice however that on Windows, `sshd_fixture.sh` is very flaky with GitHub Actions. And consistently broken for subsequent tests in AppVeyor CI: 'libssh2_session_handshake failed (-43): Failed getting banner' Another way to try is a single sshd instance serving all tests. For CMake this would probably mean using an external script. - ed25519 test keys were identical for auth and host. Regenerate the auth keypair to make them distinct. - sync the sshd environment between Docker and sshd_fixture. - use common via `openssh_server/sshd_config`. - accept same auth keys. - offer the same host keys. - sync TrustedUserCAKeys. - delete now unused keypairs: `etc/host*`, `etc/user*`. - bump up startup delay for Windows (randomly, to 5 secs, from 3). - delete `UsePrivilegeSeparation no` to avoid deprecation warnings. `command-line line 0: Deprecated option UsePrivilegeSeparation` - delete `Protocol 2` to avoid deprecation warnings. It has been the default since OpenSSH 3.0 (2001-11-06). - delete `StrictModes no` (CI tests work without it, Docker tests never used it). - bump `Dockerfile` base image to `testing-slim` (from `bullseye-slim`). It needed `sshd_config` updates to keep things working with OpenSSH 9.2 (compared to bullseye's 8.4). - replace `ChallengeResponseAuthentication` alias with `KbdInteractiveAuthentication`. The former is no longer present in default `sshd_config` since OpenSSH 8.7 (2021-08-20). This broke the `Dockerfile` script. The new name is documented since OpenSSH 4.9 (2008-03-31) - add `PubkeyAcceptedKeyTypes +ssh-rsa,ssh-dss,ssh-rsa-cert-v01@openssh.com` and `HostKeyAlgorithms +ssh-rsa`. Original-patch-by: Eric van Gyzen (@vangyzen on github) Fixes #691 There is a new name for `PubkeyAcceptedKeyTypes`: `PubkeyAcceptedAlgorithms`. It requires OpenSSH 8.5 (2021-03-03) and breaks some envs so we're not using it just yet. - drop `rijndael-cbc@lysator.liu.se` tests and references from config. This is a draft alias for `aes256-cbc`. No need to test it twice. Also this alias is no longer recognized by OpenSSH 8.5 (2021-03-03). - update `mansyntax.sh` and `sshd_fixture.sh` to not rely on `srcdir`. Hopefully this works with out-of-tree builds. - fix `test_read_algos.test` to honor CRLF EOLs in their inputs (necessary when running on Windows.) - fix `test_read_algos.test` to honor `EXEEXT`. Might be useful when running tests under cross-builds? - `test_ssh2.c`: - use libssh2 API to set blocking mode. This makes it support all platforms. - adapt socket open timeout logic from `openssh_fixture.c`. Sadly this did not help fix flakiness on GHA Windows. - tests: delete unused C headers and variable initialization. - delete unused test files: `sshd_fixture.sh.in`, `sshdwrap`, `etc/sshd_config`. Ref: cf80f2f4b5255cc85a04ee43b27a29c678c1edb1 - autotools: delete stray `.c` test sources from `EXTRA_DIST` in tests. - `tests/.gitignore`: drop two stray tests. - autotools: fix passing `SSHD` containing space (Windows needs this). - autotools: sort `EXTRA_DIST` in tests. - cmake: fix to add `test_ssh2` to `TEST_TARGETS`. - fix `authorized_key` order in `tests/gen_keys.sh`. - silence shellcheck warning in `ci/checksrc.sh`. - set `SSHD` for autotools on GitHub Actions Windows. [skipped] Auto-detection doesn't work (maybe because sshd is installed via Git for Windows and we're using MSYS2's shell.) It enables running sshd fixture (non-Docker) tests in these jobs. I did not include this in the final patch due to flakiness: ``` Connection to 127.0.0.1:4711 attempt #0 failed: retrying... Connection to 127.0.0.1:4711 attempt #1 failed: retrying... Connection to 127.0.0.1:4711 attempt #2 failed: retrying... Failure establishing SSH session: -43 ``` Can be enabled with: `export SSHD='C:/Program Files/Git/usr/bin/sshd.exe'` Closes #996 - ci: reduce algo test runtime on AppVeyor Make the block count customizable in `test_read` via environment `FIXTURE_XFER_COUNT`. Set the custom count lower than the default when running on AppVeyor. The goal is to reduce CI roundtrip times. Closes #995 GitHub (22 Apr 2023) - [Michael Buckley brought this change] Agent forwarding implementation (#752) This PR contains a series of patches that date back many years and I believe were discussed on the mailing list, but never merged. We have been using these in our local copy of libssh2 without issue since 2015, if not earlier. I believe this is the full set of changes, as we tried to use comments to mark where our copy of libssh2 differs from the canonical version. This also contains changes I made earlier this year, but which were not discussed on the mailing list, to support certificates and FIDO2 keys with agent forwarding. Note that this is not a complete implementation of agent forwarding, as that is outside the scope of libssh2. Clients still need to provide their own implementation that parses ssh-agent methods after calling libssh2_channel_read() and calls the appropriate callback messages in libssh2. See the man page changes in this PR for more details. Integration-patches-by: Viktor Szakats * prefer size_t * prefer unsigned int over u_int in public function * add const * docs, indent, checksrc, debug call, compiler warning fixes Viktor Szakats (21 Apr 2023) - ci: add Windows Server 2016 into the test mix We had Windows Server 2012 R2 (8.1) and Windows Server 2019 (10) before this patch. After, we also have Windows Server 2016 (10). The WinCNG flakey tests should have a better chance when running on the newer OS. This update does not change the compiler mix. Also change the test fixture to not use the `--quiet` option with the `docker pull` commant. This option requires docker v19.03, and AppVeyor's Visual Studio 2017 image doesn't support it. Log output did not change without `--quiet`, so it seems safe to delete it. In case we'd need it, another solution is to retry without `--quiet` if the command fails. docker's exit status is 125 in that case. Ref: https://github.com/libssh2/libssh2/issues/804#issuecomment-1515232799 Ref: https://www.appveyor.com/docs/windows-images-software/ Closes #994 - build: add autotools test_read support and more Keep a single list for mac and crypt algos that we use in both CMake and autotools. Use the same test names across build tools. Use the TAP protocol to track individual tests run from a single shell script. Also: - enable the rest of our tests with autotools. - set `make check` verbose to see errors in case they happen. - silence stray 'command not found' error when running `mansyntax.sh` on Windows. GitHub Actions Windows docker tests disabled due to: ``` Command: docker build --quiet -t libssh2/openssh_server ../tests/openssh_server Error running command 'docker build --quiet -t libssh2/openssh_server ../tests/openssh_server' (exit 1): Sending build context to Docker daemon 22.02kB Step 1/42 : FROM debian:bullseye-slim bullseye-slim: Pulling from library/debian no matching manifest for windows/amd64 10.0.20348 in the manifest list entries Failed to build docker image ``` Closes #993 - cmake: restore a dash char in comment [ci skip] It's a CMake comment header convention. GitHub (21 Apr 2023) - [Dan Fandrich brought this change] tests: add AES-GCM protocol read tests (#992) Closes #992 - [Viktor Szakats brought this change] support encrypt-then-mac (etm) MACs (#987) Support for calculating MAC (message authentication code) on encrypted data instead of plain text data. This adds support for the following MACs: - `hmac-sha1-etm@openssh.com` - `hmac-sha2-256-etm@openssh.com` - `hmac-sha2-512-etm@openssh.com` Integration-patches-by: Viktor Szakats * rebase on master * fix checksec warnings * fix compiler warning * fix indent/whitespace/eol * rebase/manual merge onto AES-GCM patch #797 * more manual merge of `libssh2_transport_send()` based on dfandrich/shellfish Fixes #582 Closes #655 Closes #987 Viktor Szakats (20 Apr 2023) - docs: fix typo in argument name [ci skip] - [Keith Dart brought this change] channel: add support for "signal" message Can send specific signals to remote process. Allows for slightly improved remote process management, if the server supports it. Integration-patches-by: Viktor Szakats * doc updates * change `signame_len` to `size_t` * variable scopes * fix checksrc warnings Closes #672 Closes #991 - crypto: add `LIBSSH2_NO_AES_CBC` option Also rename internal `LIBSSH2_AES` to `LIBSSH2_AES_CBC`. Follow-up to 857e431648df6edcb3e17138d877f2e65d2d769d Closes #990 - tidy-up: indentation fixes [ci skip] GitHub (20 Apr 2023) - [Dan Fandrich brought this change] Add support for AES-GCM crypto protocols (#797) Add support for aes256-gcm@openssh.com and aes128-gcm@openssh.com ciphers, which are the OpenSSH implementations of AES-GCM cryptography. It is similar to RFC5647 but has changes to the MAC protocol negotiation. These are implemented for recent versions of OpenSSL only. The ciphers work differently than most previous ones in two big areas: the cipher includes its own integrated MAC, and the packet length field in the SSH frame is left unencrypted. The code changes necessary are gated by flags in the LIBSSH2_CRYPT_METHOD configuration structure. These differences mean that both the first and last parts of a block require special handling during encryption. The first part is where the packet length field is, which must be kept out of the encryption path but in the authenticated part (as AAD). The last part is where the Authentication Tag is found, which is calculated and appended during encryption or removed and validated on decryption. As encryption/ decryption is performed on each packet in a loop, one block at a time, flags indicating when the first and last blocks are being processed are passed down to the encryption layers. The strict block-by-block encryption that occurs with other protocols is inappropriate for AES-GCM, since the packet length shifts the first encrypted byte 4 bytes into the block. Additionally, the final part of the block must contain the AES-GCM's Authentication Tag, so it must be presented to the lower encryption layer whole. These requirements mean added code to consolidate blocks as they are passed down. When AES-GCM is negotiated as the cipher, its built-in MAC is automatically used as the SSH MAC so further MAC negotiation is not necessary. The SSH negotiation is skipped when _libssh2_mac_override() indicates that such a cipher is in use. The virtual MAC configuration block mac_method_hmac_aesgcm is then used as the MAC placeholder. This work was sponsored by Anders Borum. Integration-patches-by: Viktor Szakats * fix checksrc errors * fix openssl.c warning * fix transport.c warnings * switch to `LIBSSH2_MIN/MAX()` from `MIN()`/`MAX()` * fix indent * fix libgcrypt unused warning * fix mbedtls unused warning * fix wincng unused warning * fix old openssl unused variable warnings * delete blank lines * updates to help merging with the ETM patch Viktor Szakats (20 Apr 2023) - tidy-up: align comments [ci skip] - tidy-up: whitespace nits [ci skip] - crypto: add/fix algo guards and extend `NO` options Add new guard `LIBSSH2_RSA_SHA1`. Add missing guards for `LIBSSH2_RSA`, `LIBSSH2_DSA`. Fix warnings when all options are disabled. This is still not complete and it's possible to break a build with certain crypto backends (e.g. mbedTLS) and/or combination of options. It's not guaranteed that all bits everywhere get disabled by these settings. Consider this a "best effort". Add these new options to disable certain crypto elements: - `LIBSSH2_NO_3DES` - `LIBSSH2_NO_AES_CTR` - `LIBSSH2_NO_BLOWFISH` - `LIBSSH2_NO_CAST` - `LIBSSH2_NO_ECDSA` - `LIBSSH2_NO_RC4` - `LIBSSH2_NO_RSA_SHA1` - `LIBSSH2_NO_RSA` The goal is to offer a way to disable legacy/obsolete/insecure ones. See also: 146a25a06dd2365a4330dad34fefcdcee1a206aa `LIBSSH2_NO_HMAC_RIPEMD` See also: 38015f4e46d8dbeea522dc7ee664522d4f47fc75 `LIBSSH2_NO_DSA` See also: be31457f3071686b555a0f0b19e5dcf63d67fc27 `LIBSSH2_NO_MD5` Closes #986 - scp: fix typo in comments [ci skip] Follow-up to 0a500b3554c29451708353279eefce750f4bca6c - base64: do not use `snprintf()` on encoding This also significantly (by 7-8x in my limited tests with a short string) speeds up this function. The impact is still minor as this function is only used in `knownhost.c` in release builds. Closes #985 - wincng: constify data arg of `libssh2_wincng_hash()` Tested in #979 - wincng: fix unused variables with `LIBSSH2_RSA_SHA2` disabled Tested in #979 - ci: delete config elements for unused 32-bit Linux builds They have been disabled since d9b4222ef1c5ab9b9e499fe6234556e5cca7c4fe Tested in #979 - ci: enable FIXTURE_TRACE_ALL_CONNECT for WinCNG tests To hopefully help finding the WinCNG hostkey verification intermittent failure #804. Tested in #979 - tests: add `FIXTURE_TRACE_ALL_CONNECT` option Works like the `FIXTURE_TRACE_ALL` envvar, but enables full trace for the connection phase only. Also fix a possible NULL deref with `FIXTURE_TRACE_ALL` and a failed `libssh2_session_init_ex()`. Tested in #979 - ci: really enable logging in AppVeyor CMake builds `CONFIGURATION` was never passed to the cmake command, so it had never enabled logging when set to `Debug`. Also `CONFIGURATION` is ambiguous depending on the "generator" used by CMake. In case of Visual Studio, this is a build/ctest-time setting, not a cmake-config parameter. So set this permanently to `Release` and enable logging via our dedicated CMake option `ENABLE_DEBUG_LOGGING`. Tested in #979 - HACKING-CRYPTO: fix stray whitespace - tidy-up: fix more nits - fix indentation errors. - reformat `cmake/FindmbedTLS.cmake` - replace a macro with a variable in `example/sftp_RW_nonblock.c`. - delete macOS macro `_DARWIN_USE_64_BIT_INODE` from the OS/400 config header, `os400/libssh2_config.h`. - fix other minor nits. Closes #983 - mansyntax: make it work on macOS, check reqs locally - use `gman` alias if present. This makes it work when the correct `man` command is provided via `brew` on macOS. - move CMake attempts to detect tools necessary to run `mansyntax.sh` into the script itself. - delete CMake TODO to move more test logic into CMake. This would make it CMake-specific and require maintaining it separately for each build tool. Just use our external script when a POSIX shell is available. Closes #982 - cmake: dedupe setting `-DHAVE_CONFIG_H` Move `libssh2_config.h` generation and setting `-DHAVE_CONFIG_H` to the root `CMakeFile.txt`. Also move symbol hiding setup there. It needs to be done before generating the config file for `LIBSSH2_API` value to be set in it. After this change the `HIDE_SYMBOLS` setting is accepted without an annoying CMake warning when not actually building a shared libssh2 lib. Closes #981 - build: assume non-blocking I/O on Windows Drop checks from Windows builds and enable it based on `WIN32`. This saves detection time and also makes 3rd party builds simpler. Also: - delete `HAVE_DISABLED_NONBLOCKING`, that we used in build tools to explicitly disable an explicit `#error` in `session.c`. - replace existing `WSAEWOULDBLOCK` check for Windows support with `WIN32`. Cleaner with the same result. Follow-up to f1e80d8d8ce9570d81836da96ba02f4d4552a7b3 Follow-up to 5644eea2161b17f7c16e18f3a10465ebb217ca1f Closes #980 - ci: rename Logging to Debug in AppVeyor - switch to internal base64 decode that uses size_t Make the public `libssh2_base64_decode()` a wrapper for that. Bump up length sizes in callers. Also fix output size calculation to first divide then multiply. Closes #978 - tests: switch to debian:bullseye-slim in Dockerfile 'slim' provides all we need, with less bloat. Tested in #976 Follow-up to 78cb64a85955f2cd9700c4fbad3f02d589dd7169 - tests: build improvements and more - rename tests to have more succint names and a more useful natural order. - rename `simple` and `ssh2` in tests to have the `test_` prefix. This avoids a name collisions with `ssh2` in examples. - cmake: drop the `example-` prefix for generated examples. Bringing their names in sync with other build tools, like autotools. - move common auth test code into the fixture and simplify tests by using that. - move feature guards from CMake to preprocessor for auth tests. Now it works with all build tools and it's easier to keep it in sync with the lib itself. For this we need to include `libssh2_priv.h` in tests, which in turn needs tweaking on the trick we use to suppress extra MSVS warnings when building tests and examples. - move mbedTLS blocklist for crypto tests from CMake to the test fixture. - add ed25519 hostkey tests to `test_hostkey` and `test_hostkey_hash`. - add shell script to regenerate all test keys used for our tests. - alpha-sort tests. - rename `signed_*` keys to begin with `key` like the rest of the keys do. - whitespace fixes. Closes #969 - autotools: rename a variable To match its counterpart we use for clang and to better match the original code in curl. Follow-up to ec0feae7920d695ce234a5aba13014bf29824c09 Closes #977 - ssh2.sh: revert likely wrong quoting [ci skip] Follow-up to 50124428509ffc2f5d08d8d3c152fa36546c9a75 - build: add `-Wbad-function-cast` picky warning Also adjust minimum gcc versions in comment. Closes #975 - tests: restore debian:bullseye in Dockerfile Follow-up to 78cb64a85955f2cd9700c4fbad3f02d589dd7169 - session: simplify preprocessor logic - by using #elif - by merging two blocks Closes #972 - tests: try debian:testing for Dockerfile Follow-up to 78cb64a85955f2cd9700c4fbad3f02d589dd7169 - src: add and use `LIBSSH2_MIN/MAX` macros Also for #797 Closes #974 - tests: switch Dockerfile to debian:testing-slim From debian:bullseye - doesn't need manual bumps. - is ahead of stable and should be stable enough for our purpose. - slim is saving resources. Closes #971 - cmake: optimize non-blocking tests on WIN32/non-WIN32 Skip testing unixy methods on Windows and vice versa. I continue to assume that CMake doesn't define `WIN32` with Cygwin (as Cygwin doesn't define `_WIN32`/`WIN32` for C), though I haven't tested this. Closes #970 GitHub (15 Apr 2023) - [Jörgen Sigvardsson brought this change] scp: option to not quote paths (#803) A new flag named `LIBSSH2_FLAG_QUOTE_PATHS` has been added, to make libssh2 not quote file paths sent to the remote's scp subsystem. Some custom ssh daemons cannot handle quoted paths, and this makes this flag useful. Authored-by: Jörgen Sigvardsson Viktor Szakats (15 Apr 2023) - cmake: make Windows builds initialize faster By skipping unixy header checks that always fail with the MSVC toolchain or all Windows toolchains. Closes #968 - cmake: use a single build rule for all tests - use the complete filename of test sources in the input list. - build all tests with the ability to access libssh2 internals. This is necessary for `test_keyboard_interactive_auth_info_request` now and might be necessary for others in the future, e.g. to avoid the depreacted public base64 decoding API. - move `test_keyboard_interactive_auth_info_request` into the main test build loop. - move `simple` into the main test build loop too. - build `ssh2` also in static mode. - cleanup the way we detect and enable gcov. - fix indentation. Closes #967 - tidy-up: more whitespace in src Closes #966 - checksrc: fix `EQUALSNULL` warnings `s/([a-z0-9._>*-]+) == NULL/!\1/g` Closes #964 - Makefile.am: add new OS400 header [ci skip] Follow-up to 6dc42e9d625deb816a051d312d09e68926959e78 - checksrc: fix `NOTEQUALSZERO` warnings Closes #963 - checksrc: fix `SIZEOFNOPAREN` warnings `s/sizeof ([a-z0-9._>*-]+)/sizeof(\1)/g` Closes #962 - crypto: add `LIBSSH2_NO_HMAC_RIPEMD` option See also: 38015f4e46d8dbeea522dc7ee664522d4f47fc75 See also: be31457f3071686b555a0f0b19e5dcf63d67fc27 Ref: https://github.com/stribika/stribika.github.io/issues/46 Closes #965 - tidy-up: example, tests continued - fix skip auth if `userauthlist` is NULL. Closes #836 (Reported-by: @sudipm-mukherjee on github) - fix most silenced `checksrc` warnings. - sync examples/tests code between each other. (output messages, error handling, declaration order, comments) - stop including unnecessary headers. - always deinitialize in case of error. - drop some redundant variables. - add error handling where missing. - show more error codes. - switch `perror()` to `fprintf()`. - fix some `printf()`s to be `fprintf()`. - formatting. Closes #960 - src: fix indentation of macro definitions (follow-up) Follow-up to d5438f4ba9036e8028f35258dd1ab97cc2edb37c - src: fix indentation of macro definitions And some comment cleanup. Closes #958 - example/ssh2_exec: drop conditional code for deprecated API GitHub (13 Apr 2023) - [monnerat brought this change] Make OS/400 implementation work again (#953) * os400: support QADRT development files in a non-standard directory This enables the possibility to compile libssh2 even if the ascii runtime development files are not installed system-wide. * userauth_kbd_packet: fix a pointer target type mismatch. A temporary variable matching the parameter type is used before copying to the real target and checking for overflow (that should not occur!). * os400qc3: move and fix big number procedures A bug added by a previous code style cleaning is fixed. _libssh2_random() now checks and return the success status. * os400qc3: fix cipher definition block lengths They were wrongly set to the key size. * Diffie-Hellman min/max modulus sizes are dependent of crypto-backend In particular, os400qc3 limits the maximum group size to 2048-bits. Move definitions of these parameters to crypto backend header files. * kex: return an error if Diffie-Hellman key pair generation fails * os400: add an ascii assert.h header file * os400qc3: implement RSA SHA2 256/512 Viktor Szakats (13 Apr 2023) - sftp: add open functions with custom attribute support Before this patch, libssh2 sent hardcoded `LIBSSH2_SFTP_ATTRIBUTES` struct on handle open. This can be problematic on some special OS, where the file size should be known on new file creation. I added two new functions to resolve this issue. Patch-by: @vajdaakos on github via #506 Changes compared to #506: - drop attr size fixup in favour of #946. - move `memcpy()` under the state where we need it. - bump filename length type to `size_t`. - fix filenames in documentation and other nits. Closes #506 Closes #947 - build: speed up and extend picky compiler options Implement picky warnings with clang in autotools. Extend picky gcc warnings, sync them between build tools and compilers and greatly speed up detection in CMake. - autotools: enable clang compiler warnings with `--enable-debug`. - autotools: enable more gcc compiler warnings with `--enable-debug`. - autotools/cmake: sync compiler warning options between gcc and clang. - sync compiler warning options between autotools and cmake. - cmake: reduce option-checks to speed up the detection phase. Bring them down to 3 (from 35). Leaving some checks to keep the CMake logic alive and for an easy way to add new options. clang 3.0 (2011-11-29) and gcc 2.95 (1999-07-31) now required. - autotools logic copied from curl, with these differences: - delete `-Wimplicit-fallthrough=4` due to a false positive. - reduce `-Wformat-truncation=2` to `1` due to a false positive. - simplify MinGW detection for `-Wno-pedantic-ms-format`. - cmake: show enabled picky compiler options (like autotools). - cmake: do compile `tests/simple.c` and `tests/ssh2.c`. - fix new compiler warnings. - `tests/CMakeLists.txt`: fix indentation. Original source of autotools logic: - https://github.com/curl/curl/blob/a8fbdb461cecbfe1ac6ecc5d8f6cf181e1507da8/acinclude.m4 - https://github.com/curl/curl/blob/a8fbdb461cecbfe1ac6ecc5d8f6cf181e1507da8/m4/curl-compilers.m4 Notice that the autotools implementation considers Apple clang as legacy clang 3.7. CMake detection works more accurately, at the same time more error-prone and difficult to update due to the sparsely documented nature of Apple clang option evolution. Closes #952 - include: delete leading underscore from macro name It can cause compiler warnings in 3rd-party code. Follow-up to 59666e03f04927e5fe3e8d8772d40729f63c570e Closes #957 - ci: use OpenSSL 3 on AppVeyor VS2022 images Closes #954 - build: be friendly with 3rd-party build tools After recent build changes, 3rd party build that took the list of C source to compile them as-is, stopped working as expected, due to `blowfish.c` and crypto-backend C sources no longer expected to compile separately but via `bcrypt_pbkdf.c` and `crypto.c`, respectively. This patch ensures that compiling these files directly result in an empty object instead of redundant code and duplicated symbols. Also: - add a compile-time error if none of the supported crypto backends are enabled. - fix `libssh2_crypto_engine()` for wolfSSL and os400qc3. Rearrange code to avoid a hard-to-find copy of crypto-backend selection guards. Follow-up to 4f0f4bff5a92dce6a6cd7a5600a8ee5660402c3f Follow-up to ff3c774e03585252b70a9ee0fcf254de7b14a767 Closes #951 - sftp: calculate attr size based on attr content in `sftp_open()` Improve robustness by replacing constant argument of `sftp_attrsize()` in `sftp_open()` with the actual `flag` value read from the `attr` we plan to transfer. Restores state of this before 37624b61e3ec4aa65a608800613d00b55ced56d7. Prerequisite for #947, #506. Also improve readability a bit and link to SFTP specs. Delete comment about version 6: The latest spec no longer features the mentioned "DO NOT IMPLEMENT" notice. Closes #946 - man: fixups - add missing `.fi` tags. - fix misplaced `.nf` tags. - add `.nf`/`.fi` tags `SYNOPSIS` where missing. - fix missing/wrong function name from `SH NAME`. - fix wrong function name in `TH`. - keep return values in a separate line. - indent. - fold long lines. - deleted `libssh2_channel_direct_streamlocal()`, there is no such function. - add missing types. - add missing headers. Closes #949 - include: indentation fixes - tidy-up: misc & minor cmake MSVS fix - `libssh2.rc`: document language/codepage codes. Ref: https://learn.microsoft.com/windows/win32/intl/code-page-identifiers - convert to Markdown: `docs/BINDINGS`, `docs/HACKING` Blind update for `vms/libssh2_make_help.dcl`. Please double-check. - cmake: fix to recognize dash-style warning options (`-Wn`) with MSVC. - `NMakefile`: sync `rd` command with `Makefile.mk`. - delete a CVS header. - cmake: simplify a `LIBSSH2_HAVE_ZLIB` macro. - few other nits and whitespace mods. Closes #943 GitHub (10 Apr 2023) - [Viktor Szakats brought this change] Support for direct-streamlocal@openssh.com UNIX socket connection (#945) This patch allow to use direct-streamlocal service from OpenSSH 6.7, that allows UNIX socket connections. Mods: - delete unrelated condition: Ref: https://github.com/libssh2/libssh2/pull/216#discussion_r374748111 - rebase on master, whitespace updates. Patch-by: @gjalves Gustavo Junior Alves Closes #216 Closes #632 Closes #945 Viktor Szakats (10 Apr 2023) - build: support `libssh2.rc` with autotools Caveat: When building `--enable-static` and `--enable-shared` at the same time, the compiled Windows resource is also included in the static library. This appears to be an autotools limitation, with no way to have different input lists (or different custom options) for shared and static libraries, even though it builds them separately. The workaround is to build static libraries in a separate `./configure` + `make` pass. Closes #944 - crypto: add `LIBSSH2_NO_DSA` to disable DSA support See also: be31457f3071686b555a0f0b19e5dcf63d67fc27 Closes #942 - build: unify source lists - introduce `src/crypto.c` as an umbrella source that does nothing else than include the selected crypto backend source. Moving this job from the built-tool to the C preprocessor. - this allows dropping the various techniques to pick the correct crypto backend sources in autotools, CMake and other build method. Including the per-backend `Makefile..inc` makefiles. - copy a trick from curl and instead of maintaining duplicate source lists for CMake, convert the GNU Makefile kept for autotools automatically. Do this in `docs`, `examples` and `src`. Ref: https://github.com/curl/curl/blob/dfabe8bca218d2524af052bd551aa87e13b8a10b/CMakeLists.txt#L1399-L1413 Also fixes missing `libssh2_setup.h` from `src/CMakeFiles.txt` after 59666e03f04927e5fe3e8d8772d40729f63c570e. - move `Makefile.inc` from root to `src`. - reformat `src/Makefile.inc` to list each source in separate lines, re-align the continuation character and sort the lists alphabetically. - update `docs/HACKING-CRYPTO` accordingly. - autotools: update the way we add crypto-backends to `LIBS`. - delete old CSV headers, indent, and merge two lines in `docs/Makefile.am` and `src/Makefile.am`. - add `libssh2.pc` to `.gitignore`, while there. Closes #941 GitHub (9 Apr 2023) - [Zenju brought this change] sftp: always clear protocol error (#787) Viktor Szakats (9 Apr 2023) - cmake: add `HIDE_SYMBOLS` option & do symbol hiding on *nix - implement symbol hiding on non-Windows platforms. The essence of the detection logic was copied from: https://github.com/curl/curl/blob/dfabe8bca218d2524af052bd551aa87e13b8a10b/CMake/CurlSymbolHiding.cmake Then simplified and shortened. This method doesn't require a recent CMake version, nor an external, auto-generated C header. Move `configure_file()` after `set(LIBSSH2_API ...)`, for the config file to pick up `LIBSSH2_API`s value. Closes #602 - add CMake option `HIDE_SYMBOLS`. This setting means to hide non-public functions from the libssh2 dynamic library when set to `ON`. The default. When set to `OFF`, make all non-static/internal functions visible in the dynamic library. This setting requires `BUILD_SHARED_LIBS=ON`. - honor this setting on Windows. By setting the `LIBSSH2_EXPORTS` manual macro again, and stop recognizing the automatic CMake macro for this purpose: `libssh2_shared_EXPORT`. Closes #939 - build: make `windows.h` even leaner Disable GDI and NLS features in `windows.h`. libssh2 doesn't use these. Closes #940 - blowfish: build improvements - include `blowfish.c` into `bcrypt_pbkdf.c`, instead of compiling it as a distinct object. - make low-level blowfish functions static. This prevents this symbols to pollute the public namespace of libssh2. It also allows the compiler to inline these functions. - integrate `blf.h` header into `bcrypt_pbkdf.c` as well. - use `_DEBUG_BLOWFISH` instead of `#if 0`. - fix `_DEBUG_BLOWFISH` compiler warnings and other nits. - `#undef` `inline` before redefining it in `libssh2_priv.h`. (copied from `blowfish.c`) - delete unused `inline` redefinitions from `blowfish.c`. - disable unused low-level blowfish functions. - formatting, header order. Closes #938 - libssh2.rc: fix debug flag, other cleanups - fix to use `LIBSSH2DEBUG` macro to set the debug flag. (was `DEBUGBUILD`, a curl-specific macro) - use manifest constants instead of literals - change language to neutral Closes #937 - tidy-up: example, tests - drop unnecessary `WIN32`-specific branches. - add `static`. - sync header inclusion order. - sync some common code between examples/tests. - fix formatting/indentation. - fix some `checksrc` errors not caught by `checksrc`. Closes #936 - tests/mansyntax.sh: avoid `if !` for portability Ref: https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Limitations-of-Builtins.html#Limitations-of-Builtins Fixes #704 Closes #935 - tidy-up: indentation in guarded #includes [ci skip] - Makefile.mk: drop `PROOT` variable [ci skip] - build: hand-crafted config rework & header tidy-up - introduce the concept of a project level setup header `src/libssh2_setup.h`, that is used by `src`, `example` and `tests` alike. Move there all common platform/compiler configuration from `src/libssh2_priv.h`, individual sources and `CMakeFiles.txt` files. Also move there our hand-crafted (= not auto-generated by CMake or autotools) configuration `win32/libssh2-config.h`. - `win32` directory is empty now, delete it. - `Makefile.mk`: adapt to the above. Build-directory is the target triplet, or any custom name set via `BLD_DIR`. - sync header path order between build systems: build/src -> source/src -> source/include - delete redundant references to `windows.h`, `winsock2.h`, `ws2tcpip.h`. - delete unnecessary #includes, update order (`libssh2_setup.h` first, `winsock2.h` first), simplify where possible. This makes the code warning-free without `WIN32_LEAN_AND_MEAN`. At the same time this patch applies this macro globally, to avoid header bloat. - example: add missing *nix header guards. - example: fix misindented `HAVE_UNISTD_H` `#ifdef`s. - set `WIN32` with all build-tools. - set `HAVE_SYS_PARAM_H` in the hand-crafted config for MinGW. To match auto-detection. - move a source-specific macro to `misc.c` from `libssh2_priv.h`. See the PR's individual commits for step-by-step updates. Closes #932 - Makefile.mk: build tests and other improvements [ci skip] - use `example` target for building examples (was: `test`). - add support for building tests via the `test` target. - accept lib-only options in a new `LIBSSH2_CPPFLAGS_LIB` variable. Useful to pass `-DLIBSSH2_EXPORTS` for correct `dllexport` in `libssh2.dll`. - fix to put dynamic library in lib directory for non-Windows builds - fix to not delete lib objects on `testclean` - test_warmup: re-implement as `test()` Instead of overriding `main()`. To align with the other tests. Overriding `main()` can cause duplicate symbols without using a lib for the `runner` code. Follow-up to 40ac6b230a309d35c57aa65a8f6d7ab6654aa3d8 Closes #934 - NMakefile: drop `/DEBUG` linker option in release mode [ci skip] - NMakefile: simplify [ci skip] - Makefile.mk: merge two rules [ci skip] - TODO: update item about compiler warnings [ci skip] Follow-up to 08354e0abbe86d4cc5088d210d53531be6d8981a Follow-up to 29347905721d2e7fbb97dabfb0071bee51db3013 Follow-up to 5a96f494ee0b00282afb2db2e091246fc5e1774a Follow-up to 463449fb9ee7dbe5fbe71a28494579a9a6890d6d Follow-up to 02f2700a61157ce5a264319bdb80754c92a40a24 GitHub (5 Apr 2023) - [ihsinme brought this change] example/x11: Add null-termination (#749) Viktor Szakats (5 Apr 2023) - crypto: fix `LIBSSH2_NO_MD5` compiler warnings Follow-up to be31457f3071686b555a0f0b19e5dcf63d67fc27 Closes #933 - build: add new man pages Follow-up to c20c81ab105cdf27f5a4e2604bd13085f46e21de GitHub (5 Apr 2023) - [Daniel Silverstone brought this change] Configurable session read timeout (#892) This set of changes provides a mechanism to runtime-configure the previously #define'd timeout for reading packets from a session. The intention here is to also extend libcurl to be able to use this interface so that when fetching from sftp servers which are very slow to return directory listings, connections do not time-out so much. * Add new field to session to hold configurable read timeout * Updated `_libssh2_packet_require()`, `_libssh2_packet_requirev()`, and `sftp_packet_requirev()` to use new field in session structure * Updated docs for API functions to set/get read timeout field in session structure * Updated `libssh2.h` to declare the get/set read timeout functions Co-authored-by: Jon Axtell Credit: Daniel Silverstone Viktor Szakats (4 Apr 2023) - cmake: whitespace fixes [ci skip] - libssh2.h: bump LIBSSH2_COPYRIGHT year [ci skip] - Makefile.mk: move portable GNU Make file to the root Move the GNU Make file formerly known as `win32/GNUmakefile` to the root directory from `win32`. It now supports any platform with a GCC-like toolchain, while also keeping support for win32. For non-Windows platforms it's necessary to provide a hand-crafted `libssh2_config.h` header for now. Usage: `make -f Makefile.mk` - src: include `limits.h` for `*_MAX` macros Follow-up to 5a96f494ee0b00282afb2db2e091246fc5e1774a Reported-by: OldWorldOrdr on github Fixes #928 Closes #930 - build: MSVS warning suppression option tidy-up - in `win32/libssh2_config.h` replace `_CRT_SECURE_NO_DEPRECATE` with `_CRT_SECURE_NO_WARNINGS`, to use the official macro for this, like in CMake. Also, it's now safe to move it back under `_MSC_VER`. Suppressing: `warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead.` `warning C4996: 'getenv': This function or variable may be unsafe. Consider using _dupenv_s instead.` - move `_CRT_NONSTDC_NO_DEPRECATE` to `example` and `tests`. Not needed for `src`. Suppressing: `warning C4996: 'strdup': The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _strdup.` `warning C4996: 'write': The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _write.` - move `_WINSOCK_DEPRECATED_NO_WARNINGS` from source files to CMake files, in `example` and `tests`. Also limit this to MSVC. Suppressing: `warning C4996: 'inet_addr': Use inet_pton() or InetPton() instead` TODO: try fixing these instead of suppressing. Closes #929 - win32/GNUmakefile: make it movable [ci skip] - add `BLD_DIR` to customize the output directory (where libs, .zip, obj subdir will go). This directory must exist. It remains `./win32` for Windows builds. - add `CONFIG_H_DIR` option to customize `libssh2_config.h` location. It remains `./win32` for Windows builds. - include `.def` in distro zip for Windows. - ready to move to the root directory. - win32/GNUmakefile: drop an unnecessary variable [ci skip] - windows: re-add `libssh2.rc` Lost while moving it from the win32 directory Follow-up to 194cfc0f84192809c87f846140e5bf06b7a864af - crypto: add `LIBSSH2_NO_MD5` to disable MD5 support Closes #927 - hostkey: fix `hash_len` field constants Replace incorrect `MD5_DIGEST_LENGTH` with `SHA_DIGEST_LENGTH` for these hostkey algos: - `ssh-rsa` and `ssh-dss` Ref: 7a5ffc8cee259bbde82ab92515cd8fea2166854b (2004-12-07 Initial) - `ssh-rsa-cert-v01@openssh.com` Ref: 4b21e49d9d2db74579b18804ed1f5eeb16578b2f (2022-07-28) Ref: #710 Also delete local fall-back definition of `MD5_DIGEST_LENGTH` (added in 9af7eb48dc3854ce8ee0589f7e2beb944e064847). Macro is no longer used. Reported-by: Markus-Schmidt on github Fixes #919 Closes #926 - ci: add MSVS 2008/2010 build tests and fix warnings Also: - fix newly surfaced (bogus) warnings in examples with MSVS 2010: ``` ..\..\example\direct_tcpip.c(262): warning C4127: conditional expression is constant ``` Happens for every `FD_SET()` macro reference. Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46677835/job/ni4hs97bh18c14ap - silence MSVS 2010 predefined Windows macro warnings: ``` ..\..\src\wincng.c(867): warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size ..\..\src\wincng.c(897): warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size ..\..\src\wincng.c(1132): warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size ``` Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46678071/job/08t5ktvkcgdghp7r Closes #925 - transport: rename local `RANDOM_PADDING` macro Rename `RANDOM_PADDING` macro used internally to enable some code. Committed in the initial version of `transport.c` in 9d55db6501aa4e21f0858cf36cdc2ddc11b96e83 (2007-02-02). libssh2 code never defined it. The name happens to collide with a Windows macro in `wincrypt.h`. `transport.c` doesn't include this header, but it includes `winsock2.h`, and it turns out it can also define this macro in some cases, e.g. when `WIN32_LEAN_AND_MEAN` is not set. To be on the safe side, prefix the name with `LIBSSH2_` to avoid enabling it by accident. Q: Maybe it'd be best to delete it with the guarded code? Reported-by: Markus-Schmidt on github Fixes #921 Closes #924 - windows: move `libssh2.rc` to the `src` directory Closes #918 - autotools: delete unused conditional `HAVE_SYS_UN_H` No longer necessary after moving the disabling/enabling logic from build tool to `example/x11.c`. Reverts 4774d500e724bc4e548f743a0cb644ab05599474 Follow-up to d245c66cc0029e480674394c23e8be1c9410f7ad - win32/GNUmakefile: update help & exit without crypto backend [ci skip] Follow-up to: 5bcd25c4c980e9765c00a2f20ac5348635063aad Follow-up to: 68fd02fba002c8c6af3ba51a2780de46b47b3787 - build: respect autotools `DLL_EXPORT` in `libssh2.h` The `DLL_EXPORT` macro is automatically set by autotools when building the libssh2 DLL. Certain toolchains might require this to correctly export symbols, so make sure to respect it in `libssh2.h` to enable `declspec(dllexport)`. With this patch we have a manual macro for that (`LIBSSH2_EXPORT`), this autotools one, the CMake one, and `_WINDLL` (added in c355d31ff94a1622526c4988b9d09074f7f7605d), possibly defined by Visual Studio. Closes #917 - build: make `HAVE_LIBCRYPT32` local to `wincng.c` libssh2 uses `wincrypt.h` aka the `crypt32` Windows system library for the function `CryptDecodeObjectEx()` [1]. This function has been available for Win32 (and UWP/WinRT apps) for a long while. Even old MinGW supports it, and also Watcom 1.9, of the rare/old compilers I checked. CMake had it permanently enabled, while it also did an extra check for the header to add the lib to the lib list. Autotools did the detection proper. Other builds had it permanently enabled. It seems safe to assume this function/header/lib is available in all environments we support. In this patch we simplify by deleting these detections and feature flags from all build tools. Keep the feature flag internal to `wincng.h`, and for extra safety add the new macro `LIBSSH2_WINCNG_DISABLE_WINCRYPT` do disable it via custom `CPPFLAGS`. WinCNG's other requirement is `bcrypt`. That also has been universally available for a long time. Here the only known outlier is old/legacy MinGW, which is missing support. [1] https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptdecodeobjectex Closes #916 - autotools: delete `src/libssh2.pc.in` reference [ci skip] Follow-up to 06f281921907fa077884c7020917661ca805b9d3 - tidy-up: null-mac/cipher documentation Move documentation for these deleted build-level options from autotools/cmake docs to the source code itself. Follow-up to 50c9bf868e833258d23c5f55ed546d1fcd5687d0 Closes #915 - cmake: re-use existing `libssh2.pc` template Instead of maintaining a second copy of `libssh2.pc.in` in `src` just for CMake, teach CMake to use the existing template in the root dir, that we already use with autotools. Closes #914 - delete redundant `HAVE_STDLIB_H` libssh2 used this standard C89 header unconditionally before this patch. Delete the feature checks and all unnecessary header guards. Closes #913 - NMakefile: drop redundant variable and assignments [ci skip] - delete redundant `HAVE_WINSOCK2_H` `libssh2.h` required `winsock2.h` for `_WIN32` since 81d53de4dc5ee39bd6215958c7dce3b12731195e (2011-06-04). Apply that to the whole codebase. This makes it unnecessary to detect `HAVE_WINSOCK2_H` and allows to drop all its uses. Completes TODO from b66d7317ca6c882afbe52fe426f68c119c40d348 TODO: Straighten out the use a mixture of `HAVE_WINDOWS_H`, `WIN32`, `_WIN32` to detect Windows. - cmake: detect WinCNG last This gives a chance to auto-detect mbedTLS on Windows with CMake. - NMakefile: rename config variables, default to WinCNG [ci skip] - replace `OPENSSLINC` and `OPENSSLLIB` with `OPENSSL_PATH`. Assume `include` and `lib` subdirs for headers and libs. - replace `WITH_ZLIB`, `ZLIBINC` and `ZLIBLIB` with `ZLIB_PATH`. Assume `include` and `lib` subdirs for header and lib. - make WinCNG the default if `WITH_OPENSSL` is not set. - win32/GNUmakefile: rename object dir and update .gitignore [ci skip] From `-{release|debug}` to `{release|debug}-`. Follow-up to 68fd02fba002c8c6af3ba51a2780de46b47b3787 - win32/GNUmakefile: add libgcrypt support [ci skip] In the previous commit 969487113aae856e43d3d905c3f2260246d44f9b, the commit message should read `win32/GNUmakefile: ` instead of `libssh2-gnumake.sh: `. Sorry for the mixup. - libssh2-gnumake.sh: make variable names platform-agnostic [ci skip] Also more consistent. Refer to DLL/SO/shared as 'dyn'. Also add comment on how to find customizable environment variables. - win32/GNUmakefile: make it support non-Windows builds [ci skip] With 20-ish extra lines, make this Makefile support all GCC-like toolchains. The temporary directory becomes `-{release|debug}` from the former `{release|debug}`. Also change the lib directory name in the `dist` package from `win32` to `lib`, to match other packages and build tools. - win32/GNUmakefile: default to WinCNG [ci skip] Also check for wolfSSL before mbedTLS to match CMake. - win32/GNUmakefile: fixups to previous commit [ci skip] - `-lws2_32` is necessary when building examples. - drop a temporary variable. Follow-up to d245c66cc0029e480674394c23e8be1c9410f7ad - delete redundant `HAVE_WS2TCPIP_H` It was used once in `src/libssh2_priv.h`, but without any effect. The header included `ws2tcpip.h` twice, once guarded by `HAVE_WS2TCPIP_H` and another time by `HAVE_WINSOCK2_H`. Dedupe these to not use `HAVE_WS2TCPIP_H`. Then delete detection of this feature from all build methods. TODO: Replace `HAVE_WINSOCK2_H` with `_WIN32`/`WIN32`. - win32/libssh2_config.h: set `HAVE_LONGLONG` & `HAVE_STDLIB_H` [ci skip] - enable `HAVE_LONGLONG` for MinGW and MSVC versions supporting it. Necessary for `GNUmakefile`/`NMakefile` builds to create the same binaries as CMake/autotools ones do. - enable `HAVE_STDLIB_H`. It has been universally available on Windows for a long time. Fixes these clang-cl warnings: ``` src\wincng.c(444,5) : warning: implicit declaration of function 'free' is invalid in C99 [-Wimplicit-function-declaration] free(buf); ^ src\wincng.c(491,20) : warning: implicitly declaring library function 'malloc' with type 'void *(unsigned long long)' [-Wimplicit-function-declaration] pbHashObject = malloc(dwHashObject); ^ src\wincng.c(491,20) : note: include the header or explicitly provide a declaration for 'malloc' src\wincng.c(2106,14) : warning: implicitly declaring library function 'realloc' with type 'void *(void *, unsigned long long)' [-Wimplicit-function-declaration] bignum = realloc(bn->bignum, length); ^ src\wincng.c(2106,14) : note: include the header or explicitly provide a declaration for 'realloc' 3 warnings generated. ``` - example: make `x11` exclusion build-tool-agnostic Whether to build the `x11` example or not was decided by each build tool. CMake didn't build it even on supported platforms. GNUMakefile used a specific blocklist for it, while autotools enabled it based on feature-detection. Migrate the enabler logic to an #ifdef in source and build `x11` unconditionally with all build tools. On unsupported platforms (=Windows) this program now displays a short message stating that fact. Also: - fix `x11.c` warnings uncovered after CMake started building it. - use `libssh2_socket_t` type for portability in `x11.c` too. - use detected header guards in `x11.c`. - delete a duplicate reference to `-lws2_32` from `win32/GNUmakefile` while there. Closes #909 - .gitignore updates [ci skip] - tidy-up: whitespace, sorting, comment and naming fixups - cmake: add missing man pages - cmake: dedupe and merge config detection Before this patch CMake did feature detections in three files: `src/CMakefiles.txt`, `examples/CMakefiles.txt` and `tests/CMakefiles.txt`. Merge and move them to the root `CMakefiles.txt`. After this patch we end up with a single `src/libssh2_config.h`. This brings CMake in sync with autotools builds, which already worked with a single config header. This also prevents mistakes where feature detection went out of sync between `src` & `tests` (see ae90a35d15d97154ac0c8554bce99ebfb18ee825). `tests` do compile sources from `src` directly, so these should always be in sync. It also allows to better integrate hand-crafted, platform-specific config headers into the builds, like the one currently residing in the `win32` directory (and also in `vms` and `os400`). Subject to an upcoming PR. Also fix a warning revealed after this patch made CMake correctly enable `HAVE_GETTIMEOFDAY` for `example` programs. Closes #906 - cmake: dedupe crypto-backend detection Before this patch CMake did crypto-backend detection in both `src/CMakefiles.txt` and `tests/CMakefiles.txt`. Merge them and move it to the root `CMakefiles.txt`. While here, also add zlib for OpenSSL. Necessary when using OpenSSL builds with zlib enabled. Closes #905 - cmake: add missing #cmakedefines to src - `HAVE_MEMSET_S` missing since 03092292597ac601c3f9f0c267ecb145dda75e4e (2018-08-02) - `HAVE_EXPLICIT_BZERO` and `HAVE_EXPLICIT_MEMSET` missing since 00005682f7b9a1aa42be50e269056ea873637047 (2023-03-28) GitHub (31 Mar 2023) - [Viktor Szakats brought this change] tidy-up: NMakefile (#903) Viktor Szakats (30 Mar 2023) - GNUmakefile: adjust win32/.gitignore [ci skip] - build: delete references to deleted NMake files [ci skip] Follow-up to 057522bb0f15c10c33159e12899ecc60e40aa6ef GitHub (30 Mar 2023) - [Viktor Szakats brought this change] NMakefile: merge them into a single file [ci skip] (#902) Also: - allow to override `AR` and `ARFLAGS`. - The extra `src` subdir in the target directory is no longer, to simplify things. - gone the dynamically generated `objects.mk`. Now replaced with some tricky logic to do that inline. - add necessary `LIBS` for WinCNG. (untested) Lightly tested via clang-cl. - [Viktor Szakats brought this change] maketgz: tidy-up [ci skip] (#901) - fix shellcheck warnings: - use quotes - use `$()` - use `printf` (instead of calling perl). - indent. - copy/adapt header comment from curl to `maketgz`. - [Viktor Szakats brought this change] ci: flatten AppVeyor jobs, add debug builds (#900) This results in better job names (now including CPU), avoiding the complex exception rules, and fine-tuning the order and variation of these tests. Enable `LIBSSH2DEBUG` for two of the existing jobs. - [Viktor Szakats brought this change] ci: add VS2022 builds (incl. ARM64) to AppVeyor (#899) - add MSVS 2022 WinCNG builds for x64 and ARM64, replacing MSVS 2013 WinCNG builds for x64 and x86. - add MSVS 2022 OpenSSL builds for x64. - fix a compiler warning uncovered by the new ARM64 build: ``` tests\openssh_fixture.c(393,17): warning C4477: 'fprintf' : format string '%d' requires an argument of type 'int', but variadic argument 1 has type 'libssh2_socket_t' tests\openssh_fixture.c(393,17): message : consider using '%lld' in the format string tests\openssh_fixture.c(393,17): message : consider using '%Id' in the format string tests\openssh_fixture.c(393,17): message : consider using '%I64d' in the format string ``` - echo the actual CMake command-line. - cmake: echo the DLL filenames found by the OpenSSL DLL-finder heuristics. - cmake: delete `libcrypto.dll` and `libssl.dll` names from the above logic. I've added these in 19884e5055b6c65f0df93d7cc776a01c518a2f06. That resulted in CMake picking up a rogue `libcrypto.dll` (with no `libssl.dll` pair) from `C:\Windows\System32\` on the `Visual Studio 2022` image, breaking tests. Turns out, OpenSSL v1.0.2 uses the "EAY" names, but let's not re-add those either, because CMake mis-picks those up from `C:/OpenSSL-Win64/bin/`, even while pointing `OPENSSL_ROOT_DIR` to a v1.1.1 installation. - cmake: set `NO_DEFAULT_PATH` for OpenSSL DLL lookup to avoid picking up all kinds of wrong DLLs. CMake considers not the first, but the _last_ hit the valid one. This happened to be `C:/Program Files/Meson/lib*-1_1.dll` when using the `Visual Studio 2022` image. Ref: https://cmake.org/cmake/help/latest/command/find_file.html - cmake: leave two commented debug lines that will be useful next time the DLL detection lookup goes wrong. Ref: https://cmake.org/cmake/help/latest/variable/CMAKE_FIND_DEBUG_MODE.html - on error, also dump `CMakeFiles/CMakeConfigureLog.yaml` if it exists (requires CMake 3.26 and newer) - [Viktor Szakats brought this change] src: fix compiler warning on Darwin (#898) ``` src/session.c:675:52: warning: implicit conversion loses integer precision: 'long' to '__darwin_suseconds_t' (aka 'int') [-Wshorten-64-to-32] tv.tv_usec = (ms_to_next - tv.tv_sec*1000) * 1000; ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~ ``` Viktor Szakats (29 Mar 2023) - tidy-up: tabs to spaces in Makefile.am [ci skip] Follow-up to 2f16d8105c9491beb2a02b3081f4f1c2a224fa62 GitHub (29 Mar 2023) - [Viktor Szakats brought this change] netware: delete support (#888) Last related commit happened 15 years ago. NetWare had it last release in 2009. All links referenced from the make file are inaccessible. - [Viktor Szakats brought this change] wolfssl: add workaround for HMAC_Update() len arg difference (#897) It's `int` in wolfSSL. `size_t` in OpenSSL/quictls/LibreSSL/BoringSSL. Ref: https://github.com/wolfSSL/wolfssl/blob/ba47562d182e10e59813da012e0ab8ef20892231/wolfssl/openssl/hmac.h#L60-L61 /cc @wolfSSL - [Viktor Szakats brought this change] cmake: introduce variables for lib target names (#896) Make our CMake config more self-documenting by introducing variables for the shared and static lib target names. Without this, it might be non-trivial to find out which line is referring to a target name vs libname, export name or other occurrences of `libssh2`. This allows to rename back the shared lib target name to the value used before 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1: `libssh2_shared` -> `libssh2`, if necessary for compatibility. Notice: before that patch, `libssh2` name referred to either the static or shared lib, depending on build settings. - [Viktor Szakats brought this change] detect and use explicit_bzero() and explicit_memset() (#895) Also skip detecting these and `memset_s()` for Windows targets in CMake, to save detection time. On Windows we always use `SecureZeroMemory()`. - [Viktor Szakats brought this change] ci: bump mbedtls (#894) - [Viktor Szakats brought this change] GNUmakefile: minor fix for DYN mode [ci skip] (#893) Follow-up to b8762c1003d97e109efa587bdc760ff9873949eb - [Viktor Szakats brought this change] build: delete MS Dev Studio build files (#891) Last updated in 2007. Also delete `VCPROJ` target remains (necessary files seem to have been missing from the repo all along) for Visual Studio 2008. Viktor Szakats (28 Mar 2023) - checksrc: fix reference in Makefile.am, update options [ci skip] GitHub (28 Mar 2023) - [Viktor Szakats brought this change] build: delete native Watcom wmake support with Win32 (#889) CMake supports generating Watcom wmake files: https://cmake.org/cmake/help/v3.1/generator/Watcom%20WMake.html - [Viktor Szakats brought this change] checksrc: update and fix warnings (#890) Update from: https://github.com/curl/curl/blob/5fec927374e4d9553205d861f2dcb39ec78002cc/scripts/checksrc.pl - suppress these new checks: - EQUALSNULL: 320 warnings - NOTEQUALSZERO: 142 warnings - TYPEDEFSTRUCT: 16 warnings We can enabled them in the future. - fix all other new ones. - also fix whitespace in two `NMakefile` files. - [Viktor Szakats brought this change] tidy-up: fix/update URLs (#887) - [Viktor Szakats brought this change] tidy-up: fix typos (#886) detected by codespell 2.2.4. - [Viktor Szakats brought this change] tidy-up: replace tabs and other whitespace (#885) There are a few non-whitespace changes, see them here: https://github.com/libssh2/libssh2/pull/885/files?w=1 - [Viktor Szakats brought this change] ci: drop cmake --parallel (#884) `--parallel 2` did not seem to make builds faster. Neither did 4 or 6. Delete this option from both GHA and AppVeyor jobs. On AppVeyor, with VS, it uses MSBuild under the hood where apparently `--parallel` doesn't do much [1]. The suggested MSBuild-specific option `/p:CL_MPcount=2` did not improve build times either. CMake spends significant time (comparable to building the project itself) on feature detection, it'd be nice to execute those in parallel, but I found not such CMake option. [1] https://discourse.cmake.org/t/parallel-does-not-really-enable-parallel-compiles-with-msbuild/964 Partial revert of 7a039d9a7a2945c10b4622f38eeed21ba6b4ec55 - [Viktor Szakats brought this change] rework how to enable insecure null-cipher/null-MAC (#873) Null-cipher and null-MAC are security footguns we want to avoid. Existing option names to toggle these were ambiguous and gave room for misinterpretation. Some projects may have had these options enabled by accident. This patch aims to make it more difficult to enable them, and making sure that existing methods require an update to stay enabled. - delete CMake/autotools settings to enable the "none" cipher and MAC. - rename existing C macros that can enable them. To use them, pass them as custom `CPPFLAGS` to the build. - enable them only if `LIBSSH2DEBUG` is also enabled. Best would be to delete them, though they may have some use while developing libssh2 itself, or debugging. - [Viktor Szakats brought this change] delete old gex (SSH2_MSG_KEX_DH_GEX_REQUEST_OLD) build option (#872) libssh2 supports an "old" style KEX message `SSH2_MSG_KEX_DH_GEX_REQUEST_OLD`, as an off-by-default build option. OpenSSH deprecated/disabled this feature in v6.9 (2015-07-01): https://www.openssh.com/releasenotes.html#6.9 This patch deletes this obsolete feature from libssh2, with no option to enable it. Added to libssh2 in: cf8ca63ea0c9388c8ae9079961d7e6a91b72b5c8 (2004-12-31) RFC: https://datatracker.ietf.org/doc/html/rfc4419 (2006-03) - [Viktor Szakats brought this change] src: more tolerant snprintf() local override (#881) `#undef snprintf` before redefining it, when `HAVE_SNPRINTF` is not defined, even though `snprintf` is available and it should have been. Possibly with 3rd party builds. Downside is that cases of missing `HAVE_SNPRINTF` are less trivially detected at compile-time. - [Viktor Szakats brought this change] ci: fix cmake warning with AppVeyor WinCNG builds (#883) ``` CMake Warning: Manually-specified variables were not used by the project: OPENSSL_ROOT_DIR ``` Follow-up to 0834b9bcc85b90c78afff103f909b5a909b95e45 - [Viktor Szakats brought this change] ci: cmake `ENABLE_WERROR` -> `ON` (#877) Consider warnings as errors for CMake jobs in CI. Viktor Szakats (26 Mar 2023) - src: silence compiler warnings 4 (alignment in WinCNG) Silence alignment warnings in WinCNG, by reworking the code. Also add two unrelated casts to avoid gcc compiler warnings in surrounding code. `increases required alignment from 1 to 4 [-Wcast-align]` `increases required alignment from 1 to 8 [-Wcast-align]` See warning details in the PR's individual commits. Reviewed-by: Marc Hörsken in Cherry-picked from #846 Closes #880 - src: silence compiler warnings 3 (change types) Apply type changes to avoid casts and warnings. In most cases this means changing to a larger type, usually `size_t` or `ssize_t`. Change signedness in a few places. Also introduce new variables to avoid reusing them for multiple purposes, to avoid casts and warnings. - add FIXME for public `libssh2_sftp_readdir_ex()` return type. - fix `_libssh2_mbedtls_rsa_sha2_verify()` to verify if `sig_len` is large enough. - fix `_libssh2_dh_key_pair()` in `wincng.c` to return error if `group_order` input is negative. Maybe we should also reject zero? - bump `_libssh2_random()` size type `int` -> `size_t`. Add checks for WinCNG and OpenSSL to return error if requested more than they support (`ULONG_MAX`, `INT_MAX` respectively). - change `_libssh2_ntohu32()` return value `unsigned int` -> `uint32_t`. - fix `_libssh2_mbedtls_bignum_random()` to check for a negative `top` input. - size down `_libssh2_wincng_key_sha_verify()` `hashlen` to match Windows'. - fix `session_disconnect()` to limit length of `lang_len` (to 256 bytes). - fix bad syntax in an `assert()`. - add a few `const` to casts. - `while(1)` -> `for(;;)`. - add casts that didn't fit into #876. - update `docs/HACKING-CRYPTO` with new sizes. May need review for OS400QC3: /cc @monnerat @jonrumsey See warning details in the PR's individual commits. Cherry-picked from #846 Closes #879 - src: silence compiler warnings 2 (ZLIB interface) Silence warnings in the ZLIB interface by adding casts and changing types. See PR for individual commits. Cherry-picked from #846 Closes #878 - src: silence compiler warnings 1 Most of the changes aim to silence warnings by adding casts. An assortment of other issues, mainly compiler warnings, resolved: - unreachable code fixed by using `goto` in `publickey_response_success()` in `publickey.c`. - potentially uninitialized variable in `sftp_open()`. - MSVS-specific bogus warnings with `nid_type` in `kex.c`. - check result of `kex_session_ecdh_curve_type()`. - add missing function declarations. - type changes to fit values without casts: - `cmd_len` in `scp_recv()` and `scp_send()`: `int` -> `size_t` - `Blowfish_expandstate()`, `Blowfish_expand0state()` loop counters: `uint16_t` -> `int` - `RECV_SEND_ALL()`: `int` -> `ssize_t` - `shell_quotearg()` -> `unsigned` -> `size_t` - `sig_len` in `_libssh2_mbedtls_rsa_sha2_sign()`: `unsigned` -> `size_t` - `prefs_len` in `libssh2_session_method_pref()`: `int` -> `size_t` - `firstsec` in `_libssh2_debug_low()`: `int` -> `long` - `method_len` in `libssh2_session_method_pref()`: `int` -> `size_t` - simplify `_libssh2_ntohu64()`. - fix `LIBSSH2_INT64_T_FORMAT` for MinGW. - fix gcc warning by not using a bit field for `burn_optimistic_kexinit`. - fix unused variable warning in `_libssh2_cipher_crypt()` in `libgcrypt.c`. - fix unused variables with `HAVE_DISABLED_NONBLOCKING`. - avoid const stripping with `BIO_new_mem_buf()` and OpenSSL 1.0.2 and newer. - add a missing const in `wincng.h`. - FIXME added for public: - `libssh2_channel_window_read_ex()` `read_avail` argument type. - `libssh2_base64_decode()` `datalen` argument type. - fix possible overflow in `sftp_read()`. Ref: 4552c73cd58fccb1fc49cb0f25f86619133e560f - formatting in `wincng.h`. See warning details in the PR's individual commits. Cherry-picked from #846 Closes #876 GitHub (24 Mar 2023) - [Viktor Szakats brought this change] cmake: automatic exports macro tidy-up (#875) In a recent CMake update I left the original CMake EXPORTS macro unchanged (`libssh2_EXPORTS`) for compatibility. However, that macro was also recently added [1] and not present in an official release yet, so we might as well just use the new native one instead (`libssh2_shared_EXPORTS`), defined by CMake automatically. This way we don't need to define the old macro manually. CMake forms this macro from the lib's internal name as defined in `add_library()` by appending `_EXPORTS`. That target name changed from `libssh2` to `libssh2_shared` after introducing dual shared + static builds in the recent update. If we're here, add a new, stable, build-tool agnostic macro with the same effect, for non-CMake use: `LIBSSH2_EXPORTS` [1] 1f0fe7443a1ecddd320f2c693607b2afee9bbe2f (2021-10-26) Follow-up to 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1 - [Viktor Szakats brought this change] maketgz: add .xz, .bz2, .zip source archive formats (#874) Copied from curl: https://github.com/curl/curl/blob/4528690cd51e5445df74aef8f83470a602683797/maketgz#L174-L222 [ci skip] Viktor Szakats (23 Mar 2023) - dist: delete reference to recently deleted file [ci skip] Follow-up to b8762c1003d97e109efa587bdc760ff9873949eb GitHub (23 Mar 2023) - [Viktor Szakats brought this change] cmake: separate compilation passes for shared/static (#871) Before this patch, cmake did a single compilation pass when we enabled both shared and static lib targets. This saves build time (esp. with MinGW targets and cross-compiling), but has the disadvantage that static libs built this way must have PIC enabled (offering slightly less performance) and `dllexport` enabled also, which means that executables linking the static libssh2 lib export its public symbols. To avoid these downsides, this patch separates the two passes and creates a non-PIC, non-`dllexport` static lib, even when also building the shared lib. - [Viktor Szakats brought this change] ci: test with OpenSSL v1.1.1 on AppVeyor (#870) Was: v1.0.2. Keep using v1.0.2 with the static-only test. To make sure we don't break support. - [Viktor Szakats brought this change] ci: speed up static-only build tests on AppVeyor (#868) - limit static-only build to a single platform (x64). - skip running ctest for the static-only build. - use MSVS 2013 for static-only builds. It's faster. - run static-only test before WinCNG ones. Otherwise it's often skipped due to WinCNG failures (#804). - [Viktor Szakats brought this change] cmake: fix error with static lib off and example/tests on (#869) Regression from 4e2580628dd1f8dc51ac65ac747ebcf0e93fa3d1 - [Viktor Szakats brought this change] ci: parallelize more (#867) - [Viktor Szakats brought this change] cmake/src: move build options before target definitions (#864) To allow more flexibility when defining targets. - [Viktor Szakats brought this change] ci: use static+shared builds to cut number of cmake jobs (#865) With CMake builds supporting static-shared libssh2 builds in a single pass, we no longer need to run static and shared jobs separately. For the same effect it's enough to run builds with both shared and static builds enabled. Halving CI jobs. We add an extra run to test the CMake config-path without shared builds enabled. This allows to add useful jobs, e.g. MSVS 2022 or ZLIB-enabled builds for Windows, valgrind builds or other useful stuff, without stretching CI run times further. Ref: #863 Viktor Szakats (22 Mar 2023) - cmake: allow building static + shared libs in a single pass - `BUILD_SHARED_LIBS=ON` no longer disables building static lib. When set, we build the static lib with PIC enabled. For shared lib only, set `BUILD_STATIC_LIBS=OFF`. For static lib without PIC, leave this option disabled. - new setting: `BUILD_STATIC_LIBS`. `ON` by default. Force-enabled when building examples or tests (we build those in static mode always.) - fix to exclude Windows Resource from the static lib. - fix to not overwrite static lib with shared implib on Windows platforms using identical suffix for them (MSVS). By using `libssh2_imp<.ext>` implib filename. - add support for `STATIC_LIB_SUFFIX` setting to set an optional suffix (e.g. `_static`) for the static lib. (experimental, not documented). Overrides the above when set. - fix to set `dllexport` when building shared lib. - set `TrackFileAccess=false` for MSVS. For faster builds, shorter verbose logs. - tests: new test linking against shared libssh2: `test_warmup_shared` - tests: simplify 'runner' lib by merging 3 libs into a single one. - tests: drop hack from `test_keyboard_interactive_auth_info_request` build. We no longer need to compile `src/misc.c` because we always link libssh2 statically. - tests: limit `FIXTURE_WORKDIR=` to the `runner` target. TL;DR: Default behavior unchanged: static (no-PIC), no shared. Enabling shared unchanged, but now also builds a static (PIC) lib by default. Based-on: b60dca8b6450a9729670986d2899cca54ccdbb6d #547 by berney on github Fixes: #547 Fixes: #675 Closes: #863 - include: silence warnings with casts in public `libssh2_sftp.h` Avoid triggering warnings in macros coming from public libssh2 headers. Cherry-picked from: #846 Closes #862 - example, tests: address compiler warnings Fix or silence all C compiler warnings discovered with (or without) `PICKY_COMPILER=ON` (in CMake). This means all warnings showing up in CI (gcc, clang, MSVS 2013/2015), in local tests on macOS (clang 14) and Windows cross-builds using gcc (12) and llvm/clang (14/15). Also fix the expression `nread -= nread` in `sftp_RW_nonblock.c`. Cherry-picked from: #846 Closes #861 - openssl: require `EVP_aes_128_ctr()` support libssh2 built with OpenSSL and without its `EVP_aes_128_ctr()`, aka `HAVE_EVP_AES_128_CTR`, option are working incorrectly. This option wasn't always auto-detected by autotools up until recently (#811). Non-cmake, non-autotools build methods never enabled it automatically. OpenSSL supports this options since at least v1.0.2, which is already EOLed and considered obsolete. OpenSSL forks (LibreSSL, BoringSSL) supported it all along. In this patch we enable this option unconditionally, now requiring OpenSSL supporting this function, or one of its forks. Also modernize OpenSSL lib references to what 1.0.2 and newer versions have been using. Fixes #739 - wincng: fix memory leak in `_libssh2_dh_secret()` Patch-by: iruis on github Assisted-by: Marc Hörsken Bug #846, commit e3487092ef9553af67633c6747cb9ab2f86465e0. Fixes #856 Closes #858 GitHub (19 Mar 2023) - [Viktor Szakats brought this change] nw, os400, watcom: stop setting unused macros [ci skip] (#859) Viktor Szakats (19 Mar 2023) - cmake: fix `ENABLE_WERROR=ON` breaking auto-detections - cmake: fix compiler warnings in `CheckNonblockingSocketSupport`. detection functions. Without this, these detections fail when `ENABLE_WERROR=ON`. - cmake: disable ENABLE_WERROR for MSVC during symbol checks in `src`. CMake's built-in symbol check function `check_symbol_exists()` generate warnings with MSVC. With warnings considered errors, these detections fail permanently. Our workaround is to disable warnings-as-errors while running these checks. ``` CheckSymbolExists.c(8): warning C4054: 'type cast': from function pointer '__int64 (__cdecl *)(const char *,char **,int)' to data pointer 'int *' in `return ((int*)(&strtoll))[argc];` ``` Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46537222/job/4vg4yg333mu2lg9b - example: replace `strcasecmp()` with C89 `strcmp()`. To avoid using CMake symbol checks in `example`. Another option is to duplicate the `check_symbol_exists()` workaround from `src`, but I figure it's not worth the complexity. We use `strcasecmp()` solely to check optional command-line options for example programs, and those are fine as lower-case. Without this, these detections fail when `ENABLE_WERROR=ON`. - also delete `__function__` detection/use in `example`. To avoid the complexity for the sake of using it at a single place in of the example's error branch. Replace that use with a literal name of the function. - cmake: also use `CMakePushCheckState` functions instead of manual save/restore. Closes #857 - build: improve a test build workaround with bcrypt - cmake: extend workaround for linking a test with shared libssh2. One of the tests uses internal libssh2 functions, and with CMake it compiles `src/misc.c` directly for this. `misc.c` references bcrypt / blowfish code. This needs a workaround for build configs where libssh2 doesn't export these. Before this patch, we enabled this workaround for MSVC. In the patch we extend this to all Windows. There is no CI test for this, but gcc and llvm/clang + mingw64 builds also need it. This may well apply to other configurations (it should, as shared libs are not supposed to export internal functions), so also make it easy to enable it at a single point. [ autotools builds force-link this one test against static libssh2. ] - make `misc.c` not depend on bcrypt. By moving out our `bcrypt_pbkdf()` wrapper into `bcrypt_pbkdf.c` itself. This allows to compile `misc.c` into tests without pulling in bcrypt / blowfish functions, and simplify the above workaround. Source code uses `HAVE_BCRYPT_PBKDF`, a leftover from original bcrypt source. We never define this inside libssh2. Defining it breaks the build, and this patch doesn't change that. - make `bcrypt_pbkdf()` static. While here, make the low-level `bcrypt_pbkdf()` function static to avoid namespace pollution. Closes #855 GitHub (17 Mar 2023) - [Viktor Szakats brought this change] ci: more timeout adjustments (#853) - add timeout to SSH connection wait loop in AppVeyor test prep. (2 minutes) - switch to per-step timeout for GitHub CI cmake/ctest runs. (10 minutes) ctest timeout (of 450 seconds) didn't seem to make any difference. Viktor Szakats (17 Mar 2023) - ci: set timeout to ctest and GitHub CI jobs - `ctest` shows a the default timeout '10000000' (turns out to be in seconds), cause infinite waits e.g. in case the necessary server worker is not available. CMake CI tests take approx: - GitHub / Linux : 125 seconds - AppVeyor / Windows: 300 seconds New timeouts are: 450 and 900 seconds respectively. - set timeouts for style-check, fuzz, Linux and Windows GitHub CI jobs to avoid hanging forever. Also: - move `choco install` to before_test to make builds start faster in `appveyor.yml`. - fix some yamllint `ON`/`OFF`-confusion issue by quoting these values in `appveyor.yml`. - fix indentation in `appveyor.yml`. - convert to GitHub workflows to LF line-ending. Ref: https://github.com/libssh2/libssh2/pull/655#issuecomment-1472853493 Closes #851 GitHub (17 Mar 2023) - [Viktor Szakats brought this change] ci: update mbedTLS repo URL, delete Travis CI (#850) Last Travis CI session run on 2021-11-18. Ref: https://app.travis-ci.com/github/libssh2/libssh2 Ref: https://travis-ci.org/github/libssh2/libssh2/builds - [Viktor Szakats brought this change] appveyor.yml: reorder tests to return relevant feedback earlier (#849) - build x64 first x64 is the more interesting target. Most type conversion issues are revealed here. Also more commonly used by now. - test VS 2013 earlier - test WinCNG earlier - delete reference to no longer used VS 2008 After this patch we end up starting with all Shared builds (2015, 2013, OpenSSL, WinCNG), then continue with Static ones. Shared/Static makes a minor if any difference in builds/tests compared to different VS versions of TLS backends. -- CI run times: Preparation + build takes: 8 x VS2015 4.5 mins -> total: 36 8 x VS2013 2 mins -> total: 16 Total: 52 mins with our 30 tests, it increases to: 8 x VS2015 8-10 mins -> total: 72 8 x VS2013 6- 9 mins -> total: 60 Total: 132 mins Without tests: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46475315 With tests: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46480549 Dan Fandrich (14 Mar 2023) - src: check for NULL pointer passed to _libssh2_get_string Callers should be protecting against this, but it's prudent to check here anyway. Fixes #802 Closes #848 Viktor Szakats (14 Mar 2023) - appveyor.yml: choco install improvements [ci skip] - avoid outputting 4000 log lines by hiding the progress bar. Reduces log size by 5x. - decrease timeout (from the default 2700 seconds). - omit unnecessary output. Tested as part of #846 GitHub (14 Mar 2023) - [Jakob Egger brought this change] build: update instructions for autoreconf (#847) The "convenience script" talks about the "buildconf" file, which is no longer recommended. - [Viktor Szakats brought this change] win32: set HAVE_STRTOLL with MSVS 2013 and newer (#845) As in curl: https://github.com/curl/curl/blob/7fa6e36583b52dd8f1e639b370c9a2849be81b54/lib/config-win32.h#L221 - [Viktor Szakats brought this change] GNUmakefile: move HAVE_STRTOLL to libssh2_config.h [ci skip] (#844) - [Viktor Szakats brought this change] src: silence unused variable warnings (#843) Viktor Szakats (13 Mar 2023) - GNUmakefile: add wolfSSL support + major rework - add wolfSSL support. - reduce size and redundant logic. - fix a bunch of small issues. - rework configuration, now with: `CC`, `AR`, `RC`, `TRIPLET`, `CFLAGS`, `CPPFLAGS`, `LDFLAGS`, `RCFLAGS`, `LIBS`, `LIBSSH2_DLL_SUFFIX`, `LIBSSH2_LDFLAGS_LIB`, `LIBSSH2_LDFLAGS_BIN` (and more). - merge examples build into the main Makefile. - relative dependency paths are now the same for building libssh2 or examples. - drop detection for obsolete OpenSSL versions (can be configure via new `OPENSSL_LIBS`). - merge dev/dist distribution zip options. - build libssh2 with `-DHAVE_STRTOLL`. - tidy-up. - build examples in static mode by default (use `DYN` to build them in shared mode). - drop forced (in non-debug mode) `-O2`. - drop Win9x support. - deprecate `ARCH` in favour of custom options and `TRIPLET`. - drop Windows resources from examples for simplicity - drop `WITH_ZLIB`. Default `ZLIB_PATH` to enable zlib support. - drop `LIBSSH2_DLL_A_SUFFIX`, use standard value `.dll` (as in `libssh2.dll.a`). - always link `bcrypt` (for LibreSSL and OpenSSL) and `crypt32` (for wolfSSL). - unhide executed build commands. - fix mbedTLS `lib` path - drop specific options to force static linking. Custom options seems a better way for this. - based on similar work made for curl: https://github.com/curl/curl/commit/a8861b6ccdd7ca35b6115588a578e36d765c9e38 Closes #842 GitHub (13 Mar 2023) - [Viktor Szakats brought this change] wincng: fix memory leak in libssh2_dh_key_pair() (#829) Fixes #722 - [Viktor Szakats brought this change] src: C89-compliant _libssh2_debug() macro (#831) Before this patch, with debug logging disabled, libssh2 code used a variadic macro to catch `_libssh2_debug()` calls, and convert them to no-ops. In certain conditions, it used an empty inline function instead. Variadic macro is a C99 feature. It means that depending on compiler, and build settings, it littered the build log with warnings about this. The new solution uses the trick of passing the variable arg list as a single argument and pass that down to the debug function with a regular macro. When disabled, another regular C89-compatible macro converts it to a no-op. This makes inlining, C99 variadic macros and maintaining the conditions for each unnecessary and also makes the codebase compile more consistently, e.g. with forced C standards and/or picky warnings. TL;DR: It makes this feature C89-compliant. - [Viktor Szakats brought this change] openssl: fix possible compiler warning in macro condition (#839) Building with wolfSSL or pre-OpenSSL v1.1.1 triggered it. ``` ../src/openssl.h:130:5: warning: 'LIBRESSL_VERSION_NUMBER' is not defined, evaluates to 0 [-Wundef] LIBRESSL_VERSION_NUMBER >= 0x3070000fL ^ ``` Regression from 2e2812dde8c1fc9b48eca592823770ab2e601f7a - [Viktor Szakats brought this change] GNUmakefile: cleanups [ci skip] (#840) - indent - sync `test/GNUmakefile` with main - delete `RANLIB` - use `else if` - use more `?=` - use ASCII-7 copyright symbol (in test) - [Viktor Szakats brought this change] win32: convert tabs to spaces [ci skip] (#838) Also strip stray newlines from `win32/rules.mk`. - [Viktor Szakats brought this change] ci: retry choco install on appveyor (#837) Trying to mitigate occasional intermittent failures while installing docker. Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46460704/job/g3t7bro6ta6n3pk6#L52 - [Viktor Szakats brought this change] cmake: drop unnecessary exception for warmup build (#835) - [Viktor Szakats brought this change] cmake: reflect minimum version in docs (#834) Follow-up to 505ea626b6e125b7ce15caf453b522192008a884 - [Viktor Szakats brought this change] cmake: add wolfSSL support to tests (#833) wolfSSL supports building with zlib as a dependency, that's the reason for the ZLIB logic in the patch. Also add it to `docs/INSTALL_CMAKE.md` and to the help text in `src/CMakeLists.txt`. Running tests not actually tested. Follow-up to 9f217a17f6f3c2047c4a1668a5c037a75a02abfd Ref: #817 - [Viktor Szakats brought this change] tests: workaround for intermittent first test failures (#832) Flakiness got continously worse these last days. It didn't seem related to recent commits. Flakiness also picked up in GitHub CI runs, something rarely seen before. Manual restart consistently fixed them. The repeating pattern was the _first_ test (`test_hostkey`) failing, with `libssh2_session_handshake failed (-13): Failed getting banner`. Failures came after a lengthy wait, suggesting a timeout. I then reversed the order of the first two tests, and it turned out that the _first_ test failed again (`test_hostkey_hash`). Also pointing to a timeout issue. Then I added a dummy test to "warm up" whatever needs warming up in the layers of CI + Docker + ssh server and their interconnects. This helped, and GitHub CI tests run without failure right for the first time. AppVeyor CI also improved a little. This patch adds a new first test called `test_warmup`, that creates a new libssh2 session, and exits with success even if that attempt failed. A stop-gap solution at best, and there is no guarantee it will continue to fix this or similar future issues, but it's also untenable to have almost every CI run fail for intermittent reasons. In some [1] cases [2] it's not the first test failing intermittently. That's a different issue, and this patch doesn't fix it. [1] #804 [2] https://ci.appveyor.com/project/libssh2org/libssh2/builds/46440828/job/8rej6cq6itg7vc4w#L500 - [Viktor Szakats brought this change] cmake: detect HAVE_SNPRINTF for tests (#830) Turns out `test_keyboard_interactive_auth_info_request.c` requires `src/libssh2_priv.h`, which in turn requires a correctly set `HAVE_SNPRINTF`. Follow-up to 4cdf785cd313c3272d04c2ef7458a35d44533d8b. - [Viktor Szakats brought this change] cmake: unset forced CMAKE_C_STANDARD 90 (#822) Added in cf80f2f4b5255cc85a04ee43b27a29c678c1edb1 (on 2016-08-14), with the title "Basic dockerised test suite". It's not clear why a C standard was explicitly set, but a side-effect of this is that CMake-built binaries diverged from ones built with autotools or GNU Make (using the same compiler and configuration). Another issue is that this may introduce ABI incompatibility with binaries built with a different C standard flag, e.g. the C compiler default or one used for other components of a final app. Seems unlikely, but if our tests require this option, we should set it for the CI builds only? - [Viktor Szakats brought this change] example: silence MSVS 2013 C4127 warnings (#828) - [Viktor Szakats brought this change] cmake: reposition ws2_32 to make binutils ld work again (#827) This restores socket libs to their pre-regression positions. Without this, `ld` doesn't find `ws2_32` symbols when referenced from TLS libs. Regression from 31fb8860dbaae3e0b7d38f2a647ee527b4b2a95f - [Viktor Szakats brought this change] fix compiling with LIBSSH2_NO_CLEAR_MEMORY and OpenSSL (#825) Regression from a0e424a51c27cc27af611ba20d134f9a9ae35273 Fixes #824 - [Viktor Szakats brought this change] snprintf: add missing prototype for local replacement (#820) Should fix these warnings with MSVS 2013 and older: `agent.c(294): warning C4013: '_libssh2_snprintf' undefined; assuming extern returning int` Follow-up to 4cdf785cd313c3272d04c2ef7458a35d44533d8b. - [Viktor Szakats brought this change] build: set _FILE_OFFSET_BITS=64 for mingw-w64 (#821) autotools builds already did auto-detect and set this mingw-specific macro, but CMake and GNU Make builds did not. This patch fixes that. Necessary for `src/scp.c`. - [Viktor Szakats brought this change] cmake: add os400qc3.c to SOURCES (#826) This re-syncs the list of compiled objects in cmake builds with non-cmake builds. Follow-up to 16619a8eddec35bb8582d1c334db0fc13b0817c4. - [Viktor Szakats brought this change] build: silence bogus C4127 warnings with MSVS 2013 and earlier (#819) E.g.: `channel.c(370): warning C4127: conditional expression is constant` Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46437333/job/5rak1vcl9hue31ei#L190 - [Viktor Szakats brought this change] cmake: use only needed socket libs when checking non-blocking sockets (#816) Based on patch by Christian Beier. Fixes #694 Closes #712 - [Viktor Szakats brought this change] cmake: update openssl dll list (#818) Add OpenSSL 3 and versionless DLL names. Also modernize warning messages and variable names. Do we need the OpenSSL-Windows-specific check and the related `RUNTIME_DEPENDENCIES` feature? The list of OpenSSL DLLs was out of date for 1.5 years without anybody noticing. Keeping it fresh is a chore and copying around DLL dependencies rarely helps as much as expected. This check also results in unuseful warnings in certain build scenarios, e.g. when linking to OpenSSL statically. - [Viktor Szakats brought this change] cmake: add wolfSSL support (#817) Implement wolfSSL support for libssh2 when building with CMake. Configuration example from curl-for-win: ``` -DCRYPTO_BACKEND=wolfSSL -DWOLFSSL_LIBRARY=/path-to/wolfssl/lib/libwolfssl.a -DWOLFSSL_INCLUDE_DIR=/path-to/wolfssl/include ``` Module `cmake/Findwolfssl.cmake` copied from: https://github.com/ngtcp2/ngtcp2/blob/e4d920c4b7a350d63b6978c68b216b76faa12635/cmake/Findwolfssl.cmake via commit: https://github.com/ngtcp2/ngtcp2/commit/296396d3730b721ad97f9de22f525400f8524c0e by Stefan Eissing - [Viktor Szakats brought this change] cmake: restore non-Windows socket lib detection (#815) I mistakenly pruned some non-Windows logic, also missing the fact that our local `check_function_exists_may_need_library()` set the `NEED_*` variables. Oddly, only `src` imported this function, yet also `examples` and `tests` called it indirectly. The referenced `HAVE_SOCKET` / `HAVE_INET_ADDR` variables might be coming from an upstream CMake project? Leaving those there also, just in case. Regression from 31fb8860dbaae3e0b7d38f2a647ee527b4b2a95f Viktor Szakats (7 Mar 2023) - build: more fixes and tidy-up (mostly for Windows) - cmake: always link `ws2_32` on Windows. Also add it to `libssh2.pc`. Fixes #745 - agent: fix gcc compiler warning: `src/agent.c:296:35: warning: 'snprintf' output truncated before the last format character [-Wformat-truncation=]` - autotools: fix `EVP_aes_128_ctr` detection with binutils `ld` The prerequisite for a successful detection is setting `LIBS=-lbcrypt` if the chosen openssl-compatible library requires it, e.g. libressl, or quictls/openssl built with `-DUSE_BCRYPTGENRANDOM`. With llvm `lld`, detection works out of the box. With binutils `ld`, it does not. The reason is `ld`s world-famous pickiness with lib order. To fix it, we pass all custom libs before and after the TLS libs. This ugly hack makes `ld` happy and detection succeed. - agent: fix Windows-specific warning: `src/agent.c:318:10: warning: implicit conversion loses integer precision: 'LRESULT' (aka 'long long') to 'int' [-Wshorten-64-to-32]` - src: fix llvm/clang compiler warning: `src/libssh2_priv.h:987:28: warning: variadic macros are a C99 feature [-Wvariadic-macros]` - src: support `inline` with `__GNUC__` (llvm/clang and gcc), fixing: ``` src/libssh2_priv.h:990:8: warning: extension used [-Wlanguage-extension-token] static inline void ^ ``` - blowfish: support `inline` keyword with MSVC. Also switch to `__inline__` (from `__inline`) for `__GNUC__`: https://gcc.gnu.org/onlinedocs/gcc/Inline.html https://clang.llvm.org/docs/UsersManual.html#differences-between-various-standard-modes - example/test: fix MSVC compiler warnings: - `example\direct_tcpip.c(209): warning C4244: 'function': conversion from 'unsigned int' to 'u_short', possible loss of data` - `tests\session_fixture.c(96): warning C4013: 'getcwd' undefined; assuming extern returning int` - `tests\session_fixture.c(100): warning C4013: 'chdir' undefined; assuming extern returning int` - delete unused macros: - `HAVE_SOCKET` - `HAVE_INET_ADDR` - `NEED_LIB_NSL` - `NEED_LIB_SOCKET` - `HAVE_NTSTATUS_H` - `HAVE_NTDEF_H` - build: delete stale zlib/openssl version numbers from path defaults. - cmake: convert tabs to spaces, add newline at EOFs. Closes #811 - cmake: make `test_read` runs cross-build-friendly Improve tests added in 7487dcf4b4ddae54b2a850737789b57b4251b0ae by running `test_read` commands directly. This makes external shell/batch files unnecessary, and is friendlier with cross-builds and when run from non-default shells, like MSYS2. Also extend CRYPT/MAC test error messages with the CRYPT/MAC name. External runner shell scripts kept for future use. Closes #814 - src: enable clear memory on all platforms - convert `_libssh2_explicit_zero()` to macro. This allows inlining where supported (e.g. `SecureZeroMemory()`). - replace `SecureZeroMemory()` (in `wincng.c`) and `LIBSSH2_CLEAR_MEMORY`-guarded `memset()` (in `os400qc3.c`) with `_libssh2_explicit_zero()` macro. - delete `LIBSSH2_CLEAR_MEMORY` guards, which enables secure-zeroing universally. - add `LIBSSH2_NO_CLEAR_MEMORY` option to disable secure-zeroing. - while here, delete double/triple inclusion of `misc.h`. `libssh2_priv.h` included it already. Closes #810 - cmake: bump minimum version to 3.1 (from 2.8.12) This allows to delete some fallback code. CMake release dates: - 2014-12-15: 3.1 - 2013-10-07: 2.8.12 Closes #813 - snprintf: unify fallback logic Before this patch, the `snprintf()` fallback logic for envs not supporting this function (i.e. Visual Studio 2013 and older) varied depending on build tool, and used different techniques in examples, tests and libssh2 itself. This patch aims to apply a common logic to libssh2 and examples/tests. - libssh2: use local `snprintf()` fallback with all build tools. We already had a local implementation, but only with CMake. Move that to the library as `_libssh2_snprintf()`, and map `snprintf()` to it when `HAVE_SNPRINTF` is not set. Also change the length type from `int` to `size_t`, and fix formatting. - set or detect `HAVE_SNPRINTF` in non-CMake builds. Detect in autotools. Keep existing logic in `win32/libssh2_config.h`. Always set for OS/400, NetWare and VMS, keeping existing behaviour. (OS/400 builds use a different local implementation) - examples/tests: drop the CMake-specific fallback logic and map `snprintf()` to `_snprintf()` for old MSVC versions, like we did before with other build tools. This is unsafe, but should be fine for these uses. - `win32/libssh2_config.h`: make it easier to read. Closes #812 - cmake: build fixes with OpenSSL/LibreSSL on Windows - Link `bcrypt` for newer (non-fork) OpenSSL. - Link `bcrypt` and `ws2_32` when using (non-fork) OpenSSL or LibreSSL, to allow `Looking for EVP_aes_128_ctr` detecting this feature. With the feature available, but not found by CMake, build failed with: `openssl.c:636:21: error: incompatible integer to pointer conversion assigning to 'EVP_CIPHER *' (aka 'struct evp_cipher_st *') from 'int' [-Wint-conversion]` Closes #809 - build fixes and improvements (mostly for Windows) - in `hostkey.c` check the result of `libssh2_sha256_init()` and `libssh2_sha512_init()` calls. This avoid the warning that we're ignoring the return values. - fix code using `int` (or `SOCKET`) for sockets. Use libssh2's dedicated `libssh2_socket_t` and `LIBSSH2_INVALID_SOCKET` instead. - fix compiler warnings due to `STATUS_*` macro redefinitions between `ntstatus.h` / `winnt.h`. Solve it by manually defining the single `STATUS` value we need from `ntstatus.h` and stop including the whole header. Fixes #733 - improve Windows UWP/WinRT builds by detecting it with code copied from the curl project. Then excluding problematic libssh2 parts according to PR by Dmitry Kostjučenko. Fixes #734 - always use `SecureZeroMemory()` on Windows. We can tweak this if not found or not inlined by a C compiler which we otherwise support. Same if it causes issues with UWP apps. Ref: https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa366877(v=vs.85) Ref: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-rtlsecurezeromemory - always enable `LIBSSH2_CLEAR_MEMORY` on Windows. CMake and curl-for-win builds already did that. Delete `SecureZeroMemory()` detection from autotools' WinCNG backend logic, that this setting used to depend on. TODO: Enable it for all platforms in a separate PR. TODO: For clearing buffers in WinCNG, call `_libssh2_explicit_zero()`, insead of a local function or explicit `SecureZeroMemory()`. - Makefile.inc: move `os400qc3.h` to `HEADERS`. This fixes compilation on non-unixy platforms. Recent regression. - `libssh2.rc`: replace copyright with plain ASCII, as in curl. Ref: curl/curl@1ca62bb Ref: curl/curl#7765 Ref: curl/curl#7776 - CMake fixes and improvements: - enable warnings with llvm/clang. - enable more comprehensive warnings with gcc and llvm/clang. Logic copied from curl: https://github.com/curl/curl/blob/233810bb5f6c5e7bedfc10bdd36607b958c0cfe4/CMakeLists.txt#L131-L148 - fix `Policy CMP0080` CMake warning by deleting that reference. - add `ENABLE_WERROR` (default: `OFF`) option. Ported from curl. - add `PICKY_COMPILER` (default: `ON`) option, as known from curl. It controls both the newly added picky warnings for llvm/clang and gcc, and also the pre-existing ones for MSVC. - `win32/GNUmakefile` fixes and improvements: - delete `_AMD64_` and add missing `-m64` for x64 builds under test. - add support for `ARCH=custom`. It disables hardcoded Intel 64-bit and Intel 32-bit options, allowing ARM64 builds. - add support for `LIBSSH2_RCFLAG_EXTRAS`. To pass custom options to windres, e.g. in ARM64 builds. - add support for `LIBSSH2_RC`. To override `windres`. - delete support for Metrowerks C. Last released in 2004. - `win32/libssh2_config.h`: delete unnecessary socket #includes `src/libssh2_priv.h` includes `winsock2.h` and `ws2tcpip.h` further down the line, triggered by `HAVE_WINSOCK2_H`. `mswsock.h` does not seem to be necessary anymore. Double-including these (before `windows.h`) caused compiler failures when building against BoringSSL and warnings with LibreSSL. We could work this around by passing `-DNOCRYPT`. Deleting the duplicates fixes these issues. Timeline: 2013: c910cd382dfa07fed2adaabf688af9e4a084fa1d deleted `mswsock.h` from `src/libssh2_priv.h` 2008: 8c43bc52b1e3de2c8fc7899a80aec0e98de4e2d8 added `winsock2.h` and `ws2tcpip.h` to `src/libssh2_priv.h` 2005: dc4bb1af967d2c53e90349f2f37324c622e714f5 added the now deleted #includes - delete or replace `LIBSSH2_WIN32` with `WIN32`. - replace hand-rolled `HAVE_WINDOWS_H` macro with `WIN32`. Also delete its detections/definitions. - delete unused `LIBSSH2_DARWIN` macro. - delete unused `writev()` Windows implementation There is no reference to `writev()` since 2007-02-02, commit 9d55db6501aa4e21f0858cf36cdc2ddc11b96e83. - fix a bunch of MSVC / llvm/clang / gcc compiler warnings: - `warning C4100: '...': unreferenced formal parameter` - using value of undefined PP macro `LIBSSH2DEBUG` - missing void from function definition - `if()` block missing in non-debug builds - unreferenced variable in non-debug builds - `warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]` in `_libssh2_debug()` - `warning C4295: 'ciphertext' : array is too small to include a terminating null character` - `warning C4706: assignment within conditional expression` - `warning C4996: 'inet_addr': Use inet_pton() or InetPton() instead or define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings` By suppressning it. Would be best to use inet_pton() as suggested. On Windows this needs Vista though. - `warning C4152: nonstandard extension, function/data pointer conversion in expression` (silenced locally) - `warning C4068: unknown pragma` Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46354480/job/j7d0m34qgq8rag5w Closes #808 Dan Fandrich (1 Mar 2023) - Add tests to check individual crypt & HMAC methods One specific crypt or hmac method is requested to be negotiated, then several MB of data is transferred. - Add test to read lots of data over a channel Connects to the ssh server then downloads several MB of data. This tests the data transfer path as well as boundary cases in packet handling as data is split into smaller SSH blocks. GitHub (27 Feb 2023) - [Will Cosgrove brought this change] Disable deprecated warnings for OpenSSL 3 #805 (#806) Disable deprecated warnings (for now) when building against OpenSSL 3 for a clean build. Reported: Daniel Stenberg Dan Fandrich (24 Feb 2023) - Fix a couple of warnings of errors in MSVC builds Two warnings (in tests & examples) in particular would cause problems: bad format causing invalid data output or a bad chdir due to out of scope buffer use. - tests: Support running tests in out-of-tree builds Various files are found by referencing the srcdir environment variable in that case. Closes #801 - Improve the ssh2 example program to run a command This performs better as an example since it shows more working code, and in the simplest possible way. It also turns the program into an actually useful tool out of the box, able to run an arbitrary command (with one restriction) on a remote machine and return the response, without needing to touch the source. Closes #800 GitHub (14 Feb 2023) - [Will Cosgrove brought this change] Add NULL session check to _libssh2_error_flags() (#796) Don't dereference null if a null session happens to make it into _libssh2_error_flags() Dan Fandrich (7 Feb 2023) - Reorder AES crypt methods so stronger ones are first This make it more likely that a stronger one will be negotiated rather than a weaker variant. - CI: update uses: dependencies to the latest versions We were seeing some deprecation warning messages on some of the older ones. - transport.c: Add some comments - Add missing files to automake makefiles & build tests Many files have been added to the cmake build files but not the automake ones in recent years. Missing ones have been added so automake "make dist" will now create a usable tar ball. The integration tests using Docker are now built with automake as well (with "make check"). They are not run yet since they aren't working yet on Linux. - tests: Fix gcc compile warnings These were mostly due to missing and non-ANSI prototypes. - Enable trace debugging in example/ssh2 This is intended to be a test program, so debugging is likely to be useful by default. - Improve example/ssh2 to allow unmodified use of public key auth The previous hard-coded key file paths were not valid for normal users. Make the paths relative to the user's home directory instead so they can work out of the box. Add a banner showing what connection will be attempted to make it easier for the user to see what is being attempted. Enable trace debugging since this is designed as a test program. GitHub (13 Dec 2022) - [Viktor Szakats brought this change] openssl.h: enable ed25519 for LibreSSL 3.7.0 (#778) This brings LibreSSL libssh2 builds on par with OpenSSL. Dan Fandrich (5 Dec 2022) - configure.ac: check for sys/param.h This file is required by glibc for the test suite. GitHub (12 Nov 2022) - [Viktor Szakats brought this change] tests: add option to run tests without docker (#762) via `export OPENSSH_NO_DOCKER=1`. SSH server host can be set via: `export OPENSSH_SERVER_HOST=127.0.0.1` SSH server port via existing: `export OPENSSH_SERVER_PORT=4711` This requires more work to be usable out of the box. The necessery sshd config is (partly) embedded into `tests/openssh_server/Dockerfile`. After this patch, it is possible to run tests in envs where docker is not installed or not available, by running a preconfigured, non-containerized sshd. - [Michael Buckley brought this change] Skip leading \r and \n characters in banner_receive() (#769) Fixes #768 Credit: Michael Buckley - [Zenju brought this change] Fixed error handling of _libssh2_packet_requirev callers (#767) Notes: some callers of _libssh2_packet_requirev() fail to set _libssh2_error(). This creates the situation where e.g. libssh2_session_handshake() fails, but libssh2_session_last_error() confusingly returns LIBSSH2_ERROR_NONE. Credit: Zenju - [Will Cosgrove brought this change] Revert usage of EVP_CipherUpdate #764 #739 (#765) Revert usage of EVP_CipherUpdate from wolfSSL PR to fix #764 #739. - [Will Cosgrove brought this change] Fix regression with rsa_sha2_verify #758 (#763) Fixes comparison with the result value coming from `mbedtls_rsa_pkcs1_verify`. Success is 0, not 1. Marc Hoersken (24 Oct 2022) - CI: fix AppVeyor status failing for starting jobs Viktor Szakats (24 Oct 2022) - delete cast5 - null-cipher mapping - more feature guard cleanup - indent - formatting - fold long lines - cleanup - temporarily silence checksrc - add mbedTLS 3.x support Make libssh2 compile cleanly with mbedTLS 3.x and later. This patch makes use of `MBEDTLS_PRIVATE()`, which is not the recommended, future-proof way to access mbedTLS data structures. This method may break with a minor upgrade, according to the authors. This is also the method used by libcurl. Also: - Fix a potentially uninitialized variable in `libssh2_mbedtls_rsa_sha2_sign()`. This happened in an error path, resulting in an unnecessary mbedTLS API call, with an uninitialized `md_type`. - Bump mbedTLS version used in CI tests to 3.2.1. Fixes #751 - tests: add option to enable all trace messages in fixture via `export FIXTURE_TRACE_ALL=1`. - win32/GNUmakefile: add mbedTLS support via `export MBEDTLS_PATH=`. Marc Hoersken (21 Oct 2022) - CI: fix AppVeyor job links only working for most recent build Ref: https://github.com/curl/curl/pull/9768#issuecomment-1286675916 Reported-by: Daniel Stenberg Follow up to #754 - CI: add missing permission section to AppVeyor status workflow Follow up to #754 - Remove OSSFuzz integration which was replaced with CIFuzz (#756) Confirmed-by: Max Dymond - Rename workflow file appveyor.yml to appveyor_docker.yml - Streamline names of CI workflow jobs - [Jeroen Ooms brought this change] Add CI for mingw-w64 via msys2 (#742) Credit: Jeroen Ooms - CI: report AppVeyor build status for each job (#754) Also give each job on AppVeyor CI a human-readable name. This aims to make job and therefore build failures more visible. GitHub (29 Sep 2022) - [Michael Buckley brought this change] Support for sk-ecdsa-sha2-nistp256 and sk-ssh-ed25519 keys, FIDO (#698) Notes: Add support for sk-ecdsa-sha2-nistp256@openssh.com and sk-ssh-ed25519@openssh.com key exchange for FIDO auth using the OpenSSL backend. Stub API for other backends. Credit: Michael Buckley - [Y. Yang brought this change] Fix DLL import library name (#711) Notes: Fix DLL import library name https://aur.archlinux.org/packages/mingw-w64-libssh2 https://cmake.org/cmake/help/latest/prop_tgt/IMPORT_PREFIX.html Credit: metab0t Y. Yang - [skundu07 brought this change] Add RSA-SHA2 support for the WinCNG backend (#736) Notes: Added code to support RSA-SHA2 for WinCNG backend. Credit: skundu07 - [Gabriel Smith brought this change] sftp: Prevent files from being skipped if the output buffer is too small (#746) Notes: LIBSSH2_ERROR_BUFFER_TOO_SMALL is returned if the buffer is too small to contain a returned directory entry. On this condition we jump to the label `end`. At this point the number of names left is decremented despite no name being returned. As suggested in #714, this commit moves the error label after the decrement of `names_left`. Fixes #714 Credit: Co-authored-by: Gabriel Smith - [bgermann brought this change] Drop advertisement clause on Blowfish (#747) Originally driven by https://github.com/pyca/bcrypt/issues/169, OpenBSD removed Niels Provos's BSD advertisement clause in version 7.1: https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/lib/libsa/blowfish.c.diff?r1=1.1&r2=1.2 https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/lib/libsa/blowfish.h.diff?r1=1.1&r2=1.2 This enables using libssh2 in GPL software. - [zhaochongliu brought this change] Support building with gcc < version 8 Files: CMakeLists.txt Notes: don't use gcc arguments that don't exist in gcc versions lower than 8 if building with older gcc. Credit: zhaochongliu - [Miguel de Icaza brought this change] Document the obscure LIBSSH2_ERROR_BAD_USE when writing to a channel (#713) Document the obscure LIBSSH2_ERROR_BAD_USE when writing to a channel Credit: Miguel de Icaza - [Michael Buckley brought this change] Don't erroneously log SSH_MSG_REQUEST_FAILURE packets from keepalive (#727) Notes: When setting a ServerAliveInterval using libssh2_keepalive_config() with want_reply set to true, some servers will reply to the keep-alive requests with a single SSH_MSG_REQUEST_FAILURE packet. This is an allowed behavior in RFC 4254, section 4. Credit: Michael Buckley - [Ryan Kelley brought this change] Updating docs for libssh2_channel_flush_ex (#728) Notes: In #614 it was identified the docs do not accurately show how libssh2_channel_flush_ex() return value is set. I have updated the doc's to correctly show what the function is returning. Credit: Ryan Kelley - [Sandeep Bansal brought this change] Support RSA certificate authentication (#710) * Adding support for signed RSA keys and unit test Credit: Sandeep Bansal Viktor Szakats (2 Jul 2022) - configure: add --disable-tests option - cmake: do not add libssh2.rc to the static library GitHub (23 May 2022) - [AyushiN brought this change] Fixed typo #697 (#701) Credit: AyushiN - [Viktor Szakats brought this change] Openssl: add support for LibreSSL 3.5.x (#700) LibreSSL 3.5.0 made more structures opaque, so let's enable existing support for that when building against these LibreSSL versions. Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt Credit: Viktor Szakats - [Michael Buckley brought this change] Ensure KEX replies don't include extra bytes (#696) Addresses #695 Credit: Michael Buckley, reported by Harry Sintonen - [Zenju brought this change] Fix buffer overflow during SSH_MSG_USERAUTH_BANNER (#693) File: userauth.c Notes: This patch fixes application crashes due to heap corruption. Turns out the null terminator is written one byte outside of the allocated area. Credit: Zenju - [Will Cosgrove brought this change] Changed NULL check to avoid logic change - [Will Cosgrove brought this change] NULL check before calling session_handshake - [Harry Sintonen brought this change] Fix build since openssl 1.1.0 when ECDSA and/or RIPEMD are disabled (#666) File: openssl.h Notes: In openssl 1.1.0 and later openssl decided to change some of the defines used to check if certain features are not compiled in the libraries. This updates the define checks. Credit: Harry Sintonen Co-authored-by: Harry Sintonen - [gbaraldi brought this change] Add RSA-SHA2 support for the mbedtls backend (#688) File: mbedtls.c Notes: * Add sha2 support for RSA key upgrading to mbedTLS backend Credit: gbaraldi Daniel Stenberg (21 Mar 2022) - misc/libssh2_copy_string: avoid malloc zero bytes Avoids the inconsistent malloc return code for malloc(0) Closes #686 Marc Hoersken (17 Mar 2022) - wincng: rename struct field referring to the DH private big number Closes #684 - tests/openssh_fixture.c: print command after variable expansion - CI: store and reuse OpenSSH Server docker image used for tests Supersedes #588 Fixes #665 Closes #685 GitHub (26 Feb 2022) - [Will Cosgrove brought this change] Added LibreSSL to crypto backend list - [Will Cosgrove brought this change] Added crypto backend list to template Added OS version as well - [Will Cosgrove brought this change] Revert "Option to build both static and shared libraries (#547)" (#675) This reverts commit b60dca8b6450a9729670986d2899cca54ccdbb6d. #547 doesn't build clean anymore with the keyboard interactive changes. - [berney brought this change] Option to build both static and shared libraries (#547) files: cmakelists.txt Notes: * Option to build both static and shared libraries when using CMake Credit: berney - [xalopp brought this change] Use modern API in userauth_keyboard_interactive() (#663) Files: userauth_kbd_packet.c, userauth_kbd_packet.h, test_keyboard_interactive_auth_info_request.c, userauth.c Notes: This refactors `SSH_MSG_USERAUTH_INFO_REQUEST` processing in `userauth_keyboard_interactive()` in order to improve robustness, correctness and readability or the code. * Refactor userauth_keyboard_interactive to use new api for packet parsing * add unit test for userauth_keyboard_interactive_parse_response() * add _libssh2_get_boolean() and _libssh2_get_byte() utility functions Credit: xalopp - [xalopp brought this change] Fix formatting in manual page (#667) Fixed formatting of `LIBSSH2_ERROR_AUTHENTICATION_FAILED` in the errors section. credit: xalopp - [tihmstar brought this change] NULL terminate server_sign_algorithms string (#669) files: packet.c, libssh2_priv.h notes: * Fix heap buffer overflow in _libssh2_key_sign_algorithm When allocating `session->server_sign_algorithms` which is a `char*` is is important to also allocate space for the string-terminating null byte at the end and make sure the string is actually null terminated. Without this fix, the `strchr()` call inside the `_libssh2_key_sign_algorithm` (line 1219) function will try to parse the string and go out of buffer on the last invocation. Credit: tihmstar Co-authored-by: Will Cosgrove - [Will Cosgrove brought this change] free RSA2 related memory (#664) Free `server_sign_algorithms` and `sign_algo_prefs`. - [Will Cosgrove brought this change] Legacy Agent support for rsa2 key upgrading/downgrading #659 (#662) Files: libssh2.h, agent.c, userauth.c Notes: Part 2 of the fix for #659. This adds rsa key downgrading for agents that don't support sha2 upgrading. It also adds better trace output for debugging/logging around key upgrading. Credit: Will Cosgrove (signed off by Michael Buckley) - [Ian Hattendorf brought this change] Support rsa-sha2 agent flags (#661) File: agent.c Notes: implements rsa-sha2 flags used to tell the agent which signing algo to use. https://tools.ietf.org/id/draft-miller-ssh-agent-01.html#rfc.section.4.5.1 Credit: Ian Hattendorf Daniel Stenberg (13 Jan 2022) - [Sunil Nimmagadda brought this change] ssh: Add support for userauth banner. The new libssh2_userauth_banner API allows to get an optional userauth banner sent with SSH_MSG_USERAUTH_BANNER packet by the server. Closes #610 GitHub (6 Jan 2022) - [Michael Buckley brought this change] Fix a memcmp errors in code that was changed from memmem to memcmp (#656) Notes: Fixed supported algo prefs list check when upgrading rsa keys Credit: Michael Buckley - [Hayden Roche brought this change] Add support for a wolfSSL crypto backend. (#629) It uses wolfSSL's OpenSSL compatibility layer, so rather than introduce new wolfssl.h/c files, the new backend just reuses openssl.h/c. Additionally, replace EVP_Cipher() calls with EVP_CipherUpdate(), since EVP_Cipher() is not recommended. Credit: Hayden Roche - [Bastien Durel brought this change] Runtime engine detection with libssh2_crypto_engine() (#643) File: version.c, HACKING-CRYPTO, libssh2.h, libssh2_crypto_engine.3, makefile. Notes: libssh2_crypto_engine() API to get crypto engine at runtime. Credit: Bastien Durel - [Will Cosgrove brought this change] RSA SHA2 256/512 key upgrade support RFC 8332 #536 (#626) Notes: * Host Key RSA 256/512 support #536 * Client side key hash upgrading for RFC 8332 * Support for server-sig-algs, ext-info-c server messages * Customizing preferred server-sig-algs via the preference LIBSSH2_METHOD_SIGN_ALGO Credit: Anders Borum, Will Cosgrove - [xalopp brought this change] fix: use userauth name length to check memory boundaries for userauth name, fixes #653 (#654) File: userauth.c Notes: Fixes `userauth_kybd_auth_name_len` length check Co-authored-by: Xaver Lopenstedt - [Daniel Stenberg brought this change] agent: handle overly large comment lengths (#651) Reported-by: Harry Sintonen - [Daniel Stenberg brought this change] userauth: check for too large userauth_kybd_auth_name_len (#650) ... before using it. Reported-by: MarcoPoloPie Fixes #649 Daniel Stenberg (17 Dec 2021) - .github/SECURITY.md: fix the URL - .github/SECURITY.md: add security policy GitHub (30 Nov 2021) - [Will Cosgrove brought this change] hostkey_method_ssh_ed25519_init() check key bounds (#645) * hostkey_method_ssh_ed25519_init() check key bounds File: hostkey.c Notes: Additional key length checking before calling _libssh2_ed25519_new_public() Credit: Will Cosgrove - [Will Cosgrove brought this change] Fix error message in memory_read_privatekey #636 file: userauth.c note: fix error message credit: volund - [cntrump brought this change] Update maketgz for macOS (#543) File: maketgz Notes: Fix error on macOS: sed: -e: No such file or directory Credit: cntrump - [Jun Tseng brought this change] CMake update minimum version to 2.8.12 (#639) File: CMakeLists.txt Notes: Following CMake's advice, Update the minimum required version. Credit: Jun Tseng Daniel Stenberg (8 Nov 2021) - [David Korczynski brought this change] ci: Add CIFuzz integration Notes: Add CIFuzz integration to run fuzzer using the OSS-Fuzz infrastructure at each PR. Signed-off-by: David Korczynski Closes #635 GitHub (26 Oct 2021) - [Uwe L. Korn brought this change] Use libssh2_EXPORTS as an alternative to _WINDLL (#470) Files: libssh2.h Notes: `_WINDLL` is only defined when a Visual Studio CMake generator is used, `libssh2_EXPORTS` is used though for all CMake generator if a shared libssh2 library is being built. Credit: Uwe L. Korn Viktor Szakats (1 Oct 2021) - windows: fix clang and WinCNG warnings Fix these categories of warning: - in `wincng.c` disagreement in signed/unsigned char when passing around the passphrase string: `warning: pointer targets in passing argument [...] differ in signedness [-Wpointer-sign]` Fixed by using `const unsigned char *` in all static functions and applying/updating casts as necessary. - in each use of `libssh2_*_init()` macros where the result is not used: `warning: value computed is not used [-Wunused-value]` Fixed by using `(void)` casts. - `channel.c:1171:7: warning: 'rc' may be used uninitialized in this function [-Wmaybe-uninitialized]` Fixed by initializing this variable with `LIBSSH2_ERROR_CHANNEL_UNKNOWN`. While there I replaced a few 0 literals with `LIBSSH2_ERROR_NONE`. - in `sftp.c`, several of these two warnings: `warning: 'data' may be used uninitialized in this function [-Wmaybe-uninitialized]` `warning: 'data_len' may be used uninitialized in this function [-Wmaybe-uninitialized]` Fixed by initializing these variables with NULL and 0 respectively. - Also removed the exec attribute from `wincng.h`. Notes: - There are many pre-existing checksrc issues. - The `sftp.c` and `channel.c` warnings may apply to other platforms as well. Closes #628 Daniel Stenberg (25 Sep 2021) - README: use www.libssh2.org for the license link - libssh2.h: bump it to 1.10.1-dev - mailing list: moved to lists.haxx.se GitHub (2 Sep 2021) - [Laurent Stacul brought this change] openssh_fixture.c: Fix openssh_server build not working (#616) (#620) File: openssh_fixture.c Notes: fixes too long of output lines building docker image Credit: Laurent Stacul - [Will Cosgrove brought this change] openssh_fixture.c: fix warning (#621) File: openssh_fixture.c Notes: Fix `portable_sleep` return type warning Credit: Will Cosgrove - [Will Cosgrove brought this change] Update CI to use latest Ubuntu #624 (#625) File: ci.yml Notes: Update CI to use latest Ubuntu #624 Also removed 32 bit building in the matrix. Credit: Will Cosgrove - [Will Cosgrove brought this change] Update .gitignore Add .DS_Store files for macOS - [Laurent Stacul brought this change] Makefile.am: Add missing key in case openssl > 1.1.0 (#617) File: Makefile.am Notes: fix missing test keys Credit: Laurent Stacul Version 1.10.0 (29 Aug 2021) Daniel Stenberg (29 Aug 2021) - [Will Cosgrove brought this change] updated docs for 1.10.0 release Marc Hörsken (30 May 2021) - [Laurent Stacul brought this change] [tests] Try several times to connect the ssh server Sometimes, as the OCI container is run in detached mode, it is possible the actual server is not ready yet to handle SSH traffic. The goal of this PR is to try several times (max 3). The mechanism is the same as for the connection to the docker machine. - [Laurent Stacul brought this change] Remove openssh_server container on test exit - [Laurent Stacul brought this change] Allow the tests to run inside a container The current tests suite starts SSH server as OCI container. This commit add the possibility to run the tests in a container provided that: * the docker client is installed builder container * the host docker daemon unix socket has been mounted in the builder container (with, if needed, the DOCKER_HOST environment variable accordingly set, and the permission to write on this socket) * the builder container is run on the default bridge network, or the host network. This PR does not handle the case where the builder container is on another network. Marc Hoersken (28 May 2021) - CI/appveyor: run SSH server for tests on GitHub Actions (#607) No longer rely on DigitalOcean to host the Docker container. Unfortunately we require a small dispatcher script that has access to a GitHub access token with scope repo in order to trigger the daemon workflow on GitHub Actions also for PRs. This script is hosted by myself for the time being until GitHub provides a tighter scope to trigger the workflow_dispatch event. GitHub (26 May 2021) - [Will Cosgrove brought this change] openssl.c: guards around calling FIPS_mode() #596 (#603) Notes: FIPS_mode() is not implemented in LibreSSL and this API is removed in OpenSSL 3.0 and was introduced in 0.9.7. Added guards around making this call. Credit: Will Cosgrove - [Will Cosgrove brought this change] configure.ac: don't undefine scoped variable (#594) * configure.ac: don't undefine scoped variable To get this script to run with Autoconf 2.71 on macOS I had to remove the undefine of the backend for loop variable. It seems scoped to the for loop and also isn't referenced later in the script so it seems OK to remove it. * configure.ac: remove cygwin specific CFLAGS #598 Notes: Remove cygwin specific Win32 CFLAGS and treat the build like a posix build Credit: Will Cosgrove, Brian Inglis - [Laurent Stacul brought this change] tests: Makefile.am: Add missing tests client keys in distribution tarball (#604) Notes: Added missing test keys. Credit: Laurent Stacul - [Laurent Stacul brought this change] Makefile.am: Add missing test keys in the distribution tarball (#601) Notes: Fix tests missing key to build the OCI image Credit: Laurent Stacul Daniel Stenberg (16 May 2021) - dist: add src/agent.h Fixes #597 Closes #599 GitHub (12 May 2021) - [Will Cosgrove brought this change] packet.c: Reset read timeout after received a packet (#576) (#586) File: packet.c Notes: Attempt keyboard interactive login (Azure AD 2FA login) and use more than 60 seconds to complete the login, the connection fails. The _libssh2_packet_require function does almost the same as _libssh2_packet_requirev but this function sets state->start = 0 before returning. Credit: teottin, Co-authored-by: Tor Erik Ottinsen - [kkoenig brought this change] Support ECDSA certificate authentication (#570) Files: hostkey.c, userauth.c, test_public_key_auth_succeeds_with_correct_ecdsa_key.c Notes: Support ECDSA certificate authentication Add a test for: - Existing ecdsa basic public key authentication - ecdsa public key authentication with a signed public key Credit: kkoenig - [Gabriel Smith brought this change] agent.c: Add support for Windows OpenSSH agent (#517) Files: agent.c, agent.h, agent_win.c Notes: * agent: Add support for Windows OpenSSH agent The implementation was partially taken and modified from that found in the Portable OpenSSH port to Win32 by the PowerShell team, but mostly based on the existing Unix OpenSSH agent support. https://github.com/PowerShell/openssh-portable Regarding the partial transfer support implementation: partial transfers are easy to deal with, but you need to track additional state when non-blocking IO enters the picture. A tracker of how many bytes have been transfered has been placed in the transfer context struct as that's where it makes most sense. This tracker isn't placed behind a WIN32 #ifdef as it will probably be useful for other agent implementations. * agent: win32 openssh: Disable overlapped IO Non-blocking IO is not currently supported by the surrounding agent code, despite a lot of the code having everything set up to handle it. Credit: Co-authored-by: Gabriel Smith - [Zenju brought this change] Fix detailed _libssh2_error being overwritten (#473) Files: openssl.c, pem.c, userauth.c Notes: * Fix detailed _libssh2_error being overwritten by generic errors * Unified error handling Credit: Zenju - [Paul Capron brought this change] Fix _libssh2_random() silently discarding errors (#520) Notes: * Make _libssh2_random return code consistent Previously, _libssh2_random was advertized in HACKING.CRYPTO as returning `void` (and was implemented that way in os400qc3.c), but that was in other crypto backends a lie; _libssh2_random is (a macro expanding) to an int-value expression or function. Moreover, that returned code was: — 0 or success, -1 on error for the MbedTLS & WinCNG crypto backends But also: — 1 on success, -1 or 0 on error for the OpenSSL backend! – 1 on success, error cannot happen for libgcrypt! This commit makes explicit that _libssh2_random can fail (because most of the underlying crypto functions can indeed fail!), and it makes its result code consistent: 0 on success, -1 on error. This is related to issue #519 https://github.com/libssh2/libssh2/issues/519 It fixes the first half of it. * Don't silent errors of _libssh2_random Make sure to check the returned code of _libssh2_random(), and propagates any failure. A new LIBSSH_ERROR_RANDGEN constant is added to libssh2.h None of the existing error constants seemed fit. This commit is related to d74285b68450c0e9ea6d5f8070450837fb1e74a7 and to https://github.com/libssh2/libssh2/issues/519 (see the issue for more info.) It closes #519. Credit: Paul Capron - [Gabriel Smith brought this change] ci: Remove caching of docker image layers (#589) Notes: continued ci reliability work. Credit: Gabriel Smith - [Gabriel Smith brought this change] ci: Speed up docker builds for tests (#587) Notes: The OpenSSH server docker image used for tests is pre-built to prevent wasting time building it during a test, and unneeded rebuilds are prevented by caching the image layers. Credit: Gabriel Smith - [Will Cosgrove brought this change] userauth.c: don't error if using keys without RSA (#555) file: userauth.c notes: libssh2 now supports many other key types besides RSA, if the library is built without RSA support and a user attempts RSA auth it shouldn't be an automatic error credit: Will Cosgrove - [Marc brought this change] openssl.c: Avoid OpenSSL latent error in FIPS mode (#528) File: openssl.c Notes: Avoid initing MD5 digest, which is not permitted in OpenSSL FIPS certified cryptography mode. Credit: Marc - [Laurent Stacul brought this change] openssl.c: Fix EVP_Cipher interface change in openssl 3 #463 File: openssl.c Notes: Fixes building with OpenSSL 3, #463. The change is described there: https://github.com/openssl/openssl/commit/f7397f0d58ce7ddf4c5366cd1846f16b341fbe43 Credit: Laurent Stacul, reported by Sergei - [Gabriel Smith brought this change] openssh_fixture.c: Fix potential overwrite of buffer when reading stdout of command (#580) File: openssh_fixture.c Notes: If reading the full output from the executed command took multiple passes (such as when reading multiple lines) the old code would read into the buffer starting at the some position (the start) every time. The old code only works if fgets updated p or had an offset parameter, both of which are not true. Credit: Gabriel Smith - [Gabriel Smith brought this change] ci: explicitly state the default branch (#585) Notes: It looks like the $default-branch macro only works in templates, not workflows. This is not explicitly stated anywhere except the linked PR comment. https://github.com/actions/starter-workflows/pull/590#issuecomment-672360634 credit: Gabriel Smith - [Gabriel Smith brought this change] ci: Swap from Travis to Github Actions (#581) Files: ci files Notes: Move Linux CI using Github Actions Credit: Gabriel Smith, Marc Hörsken - [Mary brought this change] libssh2_priv.h: add iovec on 3ds (#575) file: libssh2_priv.h note: include iovec for 3DS credit: Mary Mstrodl - [Laurent Stacul brought this change] Tests: Fix unused variables warning (#561) file: test_public_key_auth_succeeds_with_correct_ed25519_key_from_mem.c notes: fixed unused vars credit: Laurent Stacul - [Viktor Szakats brought this change] bcrypt_pbkdf.c: fix clang10 false positive warning (#563) File: bcrypt_pbkdf.c Notes: blf_enc() takes a number of 64-bit blocks to encrypt, but using sizeof(uint64_t) in the calculation triggers a warning with clang 10 because the actual data type is uint32_t. Pass BCRYPT_BLOCKS / 2 for the number of blocks like libc bcrypt(3) does. Ref: https://github.com/openbsd/src/commit/04a2240bd8f465bcae6b595d912af3e2965856de Fixes #562 Credit: Viktor Szakats - [Will Cosgrove brought this change] transport.c: release payload on error (#554) file: transport.c notes: If the payload is invalid and there is an early return, we could leak the payload credit: Will Cosgrove - [Will Cosgrove brought this change] ssh2_client_fuzzer.cc: fixed building The GitHub web editor did some funky things - [Will Cosgrove brought this change] ssh_client_fuzzer.cc: set blocking mode on (#553) file: ssh_client_fuzzer.cc notes: the session needs blocking mode turned on to avoid EAGAIN being returned from libssh2_session_handshake() credit: Will Cosgrove, reviewed by Michael Buckley - [Etienne Samson brought this change] Add a LINT option to CMake (#372) * ci: make style-checking available locally * cmake: add a linting target * tests: check test suite syntax with checksrc.pl - [Will Cosgrove brought this change] kex.c: kex_agree_instr() improve string reading (#552) * kex.c: kex_agree_instr() improve string reading file: kex.c notes: if haystack isn't null terminated we should use memchr() not strchar(). We should also make sure we don't walk off the end of the buffer. credit: Will Cosgrove, reviewed by Michael Buckley - [Will Cosgrove brought this change] kex.c: use string_buf in ecdh_sha2_nistp (#551) * kex.c: use string_buf in ecdh_sha2_nistp file: kex.c notes: use string_buf in ecdh_sha2_nistp() to avoid attempting to parse malformed data - [Will Cosgrove brought this change] kex.c: move EC macro outside of if check #549 (#550) File: kex.c Notes: Moved the macro LIBSSH2_KEX_METHOD_EC_SHA_HASH_CREATE_VERIFY outside of the LIBSSH2_ECDSA since it's also now used by the ED25519 code. Sha 256, 384 and 512 need to be defined for all backends now even if they aren't used directly. I believe this is already the case, but just a heads up. Credit: Stefan-Ghinea - [Tim Gates brought this change] kex.c: fix simple typo, niumber -> number (#545) File: kex.c Notes: There is a small typo in src/kex.c. Should read `number` rather than `niumber`. Credit: Tim Gates - [Tseng Jun brought this change] session.c: Correct a typo which may lead to stack overflow (#533) File: session.c Notes: Seems the author intend to terminate banner_dup buffer, later, print it to the debug console. Author: Tseng Jun Marc Hoersken (10 Oct 2020) - wincng: fix random big number generation to match openssl The old function would set the least significant bits in the most significant byte instead of the most significant bits. The old function would also zero pad too much bits in the most significant byte. This lead to a reduction of key space in the most significant byte according to the following listing: - 8 bits reduced to 0 bits => eg. 2048 bits to 2040 bits DH key - 7 bits reduced to 1 bits => eg. 2047 bits to 2041 bits DH key - 6 bits reduced to 2 bits => eg. 2046 bits to 2042 bits DH key - 5 bits reduced to 3 bits => eg. 2045 bits to 2043 bits DH key No change would occur for the case of 4 significant bits. For 1 to 3 significant bits in the most significant byte the DH key would actually be expanded instead of reduced: - 3 bits expanded to 5 bits => eg. 2043 bits to 2045 bits DH key - 2 bits expanded to 6 bits => eg. 2042 bits to 2046 bits DH key - 1 bits expanded to 7 bits => eg. 2041 bits to 2047 bits DH key There is no case of 0 significant bits in the most significant byte since this would be a case of 8 significant bits in the next byte. At the moment only the following case applies due to a fixed DH key size value currently being used in libssh2: The DH group_order is fixed to 256 (bytes) which leads to a 2047 bits DH key size by calculating (256 * 8) - 1. This means the DH keyspace was previously reduced from 2047 bits to 2041 bits (while the top and bottom bits are always set), so the keyspace is actually always reduced from 2045 bits to 2039 bits. All of this is only relevant for Windows versions supporting the WinCNG backend (Vista or newer) before Windows 10 version 1903. Closes #521 Daniel Stenberg (28 Sep 2020) - libssh2_session_callback_set.3: explain the recv/send callbacks Describe how to actually use these callbacks. Closes #518 GitHub (23 Sep 2020) - [Will Cosgrove brought this change] agent.c: formatting Improved formatting of RECV_SEND_ALL macro. - [Will Cosgrove brought this change] CMakeLists.txt: respect install lib dir #405 (#515) Files: CMakeLists.txt Notes: Use CMAKE_INSTALL_LIBDIR directory Credit: Arfrever - [Will Cosgrove brought this change] kex.c: group16-sha512 and group18-sha512 support #457 (#468) Files: kex.c Notes: Added key exchange group16-sha512 and group18-sha512. As a result did the following: Abstracted diffie_hellman_sha256() to diffie_hellman_sha_algo() which is now algorithm agnostic and takes the algorithm as a parameter since we needed sha512 support. Unfortunately it required some helper functions but they are simple. Deleted diffie_hellman_sha1() Deleted diffie_hellman_sha1 specific macro Cleaned up some formatting Defined sha384 in os400 and wincng backends Defined LIBSSH2_DH_MAX_MODULUS_BITS to abort the connection if we receive too large of p from the server doing sha1 key exchange. Reorder the default key exchange list to match OpenSSH and improve security Credit: Will Cosgrove - [Igor Klevanets brought this change] agent.c: Recv and send all bytes via network in agent_transact_unix() (#510) Files: agent.c Notes: Handle sending/receiving partial packet replies in agent.c API. Credit: Klevanets Igor - [Daniel Stenberg brought this change] Makefile.am: include all test files in the dist #379 File: Makefile.am Notes: No longer conditionally include OpenSSL specific test files, they aren't run if we're not building against OpenSSL 1.1.x anyway. Credit: Daniel Stenberg - [Max Dymond brought this change] Add support for an OSS Fuzzer fuzzing target (#392) Files: .travis.yml, configure.ac, ossfuzz Notes: This adds support for an OSS-Fuzz fuzzing target in ssh2_client_fuzzer, which is a cut down example of ssh2.c. Future enhancements can improve coverage. Credit: Max Dymond - [Sebastián Katzer brought this change] mbedtls.c: ECDSA support for mbed TLS (#385) Files: mbedtls.c, mbedtls.h, .travis.yml Notes: This PR adds support for ECDSA for both key exchange and host key algorithms. The following elliptic curves are supported: 256-bit curve defined by FIPS 186-4 and SEC1 384-bit curve defined by FIPS 186-4 and SEC1 521-bit curve defined by FIPS 186-4 and SEC1 Credit: Sebastián Katzer Marc Hoersken (1 Sep 2020) - buildconf: exec autoreconf to avoid additional process (#512) Also make buildconf exit with the return code of autoreconf. Follow up to #224 - scp.c: fix indentation in shell_quotearg documentation - wincng: make more use of new helper functions (#496) - wincng: make sure algorithm providers are closed once (#496) GitHub (10 Jul 2020) - [David Benjamin brought this change] openssl.c: clean up curve25519 code (#499) File: openssl.c, openssl.h, crypto.h, kex.c Notes: This cleans up a few things in the curve25519 implementation: - There is no need to create X509_PUBKEYs or PKCS8_PRIV_KEY_INFOs to extract key material. EVP_PKEY_get_raw_private_key and EVP_PKEY_get_raw_public_key work fine. - libssh2_x25519_ctx was never used (and occasionally mis-typedefed to libssh2_ed25519_ctx). Remove it. The _libssh2_curve25519_new and _libssh2_curve25519_gen_k interfaces use the bytes. Note, if it needs to be added back, there is no need to roundtrip through EVP_PKEY_new_raw_private_key. EVP_PKEY_keygen already generated an EVP_PKEY. - Add some missing error checks. Credit: David Benjamin - [Will Cosgrove brought this change] transport.c: socket is disconnected, return error (#500) File: transport.c Notes: This is to fix #102, instead of continuing to attempt to read a disconnected socket, it will now error out. Credit: TDi-jonesds - [Will Cosgrove brought this change] stale.yml Increasing stale values. Marc Hoersken (6 Jul 2020) - wincng: try newer DH API first, fallback to legacy RSA API Avoid the use of RtlGetVersion or similar Win32 functions, since these depend on version information from manifests. This commit makes the WinCNG backend first try to use the new DH algorithm API with the raw secret derivation feature. In case this feature is not available the WinCNG backend will fallback to the classic approach of using RSA-encrypt to perform the required modular exponentiation of BigNums. The feature availability test is done during the first handshake and the result is stored in the crypto backends global state. Follow up to #397 Closes #484 - wincng: fix indentation of function arguments and comments Follow up to #397 - [Wez Furlong brought this change] wincng: use newer DH API for Windows 8.1+ Since Windows 1903 the approach used to perform DH kex with the CNG API has been failing. This commit switches to using the `DH` algorithm provider to perform generation of the key pair and derivation of the shared secret. It uses a feature of CNG that is not yet documented. The sources of information that I've found on this are: * https://stackoverflow.com/a/56378698/149111 * https://github.com/wbenny/mini-tor/blob/5d39011e632be8e2b6b1819ee7295e8bd9b7a769/mini/crypto/cng/dh.inl#L355 With this change I am able to successfully connect from Windows 10 to my ubuntu system. Refs: https://github.com/alexcrichton/ssh2-rs/issues/122 Fixes: https://github.com/libssh2/libssh2/issues/388 Closes: https://github.com/libssh2/libssh2/pull/397 GitHub (1 Jul 2020) - [Zenju brought this change] comp.c: Fix name clash with ZLIB macro "compress" (#418) File: comp.c Notes: * Fix name clash with ZLIB macro "compress". Credit: Zenju - [yann-morin-1998 brought this change] buildsystem: drop custom buildconf script, rely on autoreconf (#224) Notes: The buildconf script is currently required, because we need to copy a header around, because it is used both from the library and the examples sources. However, having a custom 'buildconf'-like script is not needed if we can ensure that the header exists by the time it is needed. For that, we can just append the src/ directory to the headers search path for the examples. And then it means we no longer need to generate the same header twice, so we remove the second one from configure.ac. Now, we can just call "autoreconf -fi" to generate the autotools files, instead of relying on the canned sequence in "buildconf", since autoreconf has now long known what to do at the correct moment (future versions of autotools, automake, autopoint, autoheader etc... may require an other ordering, or other intermediate steps, etc...). Eventually, get rid of buildconf now it is no longer needed. In fact, we really keep it for legacy, but have it just call autoreconf (and print a nice user-friendly warning). Don't include it in the release tarballs, though. Update doc, gitignore, and travis-CI jobs accordingly. Credit: Signed-off-by: "Yann E. MORIN" Cc: Sam Voss - [Will Cosgrove brought this change] libssh2.h: Update Diffie Hellman group values (#493) File: libssh2.h Notes: Update the min, preferred and max DH group values based on RFC 8270. Credit: Will Cosgrove, noted from email list by Mitchell Holland Marc Hoersken (22 Jun 2020) - travis: use existing Makefile target to run checksrc - Makefile: also run checksrc on test source files - tests: avoid use of deprecated function _sleep (#490) - tests: avoid use of banned function strncat (#489) - tests: satisfy checksrc regarding max line length of 79 chars Follow up to 2764bc8e06d51876b6796d6080c6ac51e20f3332 - tests: satisfy checksrc with whitespace only fixes checksrc.pl -i4 -m79 -ASIZEOFNOPAREN -ASNPRINTF -ACOPYRIGHT -AFOPENMODE tests/*.[ch] - tests: add support for ports published via Docker for Windows - tests: restore retry behaviour for docker-machine ip command - tests: fix mix of declarations and code failing C89 compliance - wincng: add and improve checks in bit counting function - wincng: align bits to bytes calculation in all functions - wincng: do not disable key validation that can be enabled The modular exponentiation also works with key validation enabled. - wincng: fix return value in _libssh2_dh_secret Do not ignore return value of modular exponentiation. - appveyor: build and run tests for WinCNG crypto backend GitHub (1 Jun 2020) - [suryakalpo brought this change] INSTALL_CMAKE.md: Update formatting (#481) File: INSTALL_CMAKE.md Notes: Although the original text would be immediately clear to seasoned users of CMAKE and/or Unix shell, the lack of newlines may cause some confusion for newcomers. Hence, wrapping the texts in a md code-block such that the newlines appear as intended. credit: suryakalpo Marc Hoersken (31 May 2020) - src: add new and align include guards in header files (#480) Make sure all include guards exist and follow the same format. - wincng: fix multiple definition of `_libssh2_wincng' (#479) Add missing include guard and move global state from header to source file by using extern. GitHub (28 May 2020) - [Will Cosgrove brought this change] transport.c: moving total_num check from #476 (#478) file: transport.c notes: moving total_num zero length check from #476 up to the prior bounds check which already includes a total_num check. Makes it slightly more readable. credit: Will Cosgrove - [lutianxiong brought this change] transport.c: fix use-of-uninitialized-value (#476) file:transport.c notes: return error if malloc(0) credit: lutianxiong - [Dr. Koutheir Attouchi brought this change] libssh2_sftp.h: Changed type of LIBSSH2_FX_* constants to unsigned long, fixes #474 File: libssh2_sftp.h Notes: Error constants `LIBSSH2_FX_*` are only returned by `libssh2_sftp_last_error()` which returns `unsigned long`. Therefore these constants should be defined as unsigned long literals, instead of int literals. Credit: Dr. Koutheir Attouchi - [monnerat brought this change] os400qc3.c: constify libssh2_os400qc3_hash_update() data parameter. (#469) Files: os400qc3.c, os400qc3.h Notes: Fixes building on OS400. #426 Credit: Reported-by: hjindra on github, dev by Monnerat - [monnerat brought this change] HACKING.CRYPTO: keep up to date with new crypto definitions from code. (#466) File: HACKING.CRYPTO Notes: This commit updates the HACKING.CRYPTO documentation file in an attempt to make it in sync with current code. New documented features are: SHA384 SHA512 ECDSA ED25519 Credit: monnerat - [Harry Sintonen brought this change] kex.c: Add diffie-hellman-group14-sha256 Key Exchange Method (#464) File: kex.c Notes: Added diffie-hellman-group14-sha256 kex Credit: Harry Sintonen - [Will Cosgrove brought this change] os400qc3.h: define sha512 macros (#465) file: os400qc3.h notes: fixes for building libssh2 1.9.x - [Will Cosgrove brought this change] os400qc3.h: define EC types to fix building #426 (#462) File: os400qc3.h Notes: define missing EC types which prevents building Credit: hjindra - [Brendan Shanks brought this change] hostkey.c: Fix 'unsigned int'/'uint32_t' mismatch (#461) File: hostkey.c Notes: These types are the same size so most compilers are fine with it, but CodeWarrior (on classic MacOS) throws an ‘illegal implicit conversion’ error Credit: Brendan Shanks - [Thomas Klausner brought this change] Makefile.am: Fix unportable test(1) operator. (#459) file: Makefile.am Notes: The POSIX comparison operator for test(1) is =; bash supports == but not even test from GNU coreutils does. Credit: Thomas Klausner - [Tseng Jun brought this change] openssl.c: minor changes of coding style (#454) File: openssl.c Notes: minor changes of coding style and align preprocessor conditional for #439 Credit: Tseng Jun - [Hans Meier brought this change] openssl.c: Fix for use of uninitialized aes_ctr_cipher.key_len (#453) File: Openssl.c Notes: * Fix for use of uninitialized aes_ctr_cipher.key_len when using HAVE_OPAQUE_STRUCTS, regression from #439 Credit: Hans Meirer, Tseng Jun - [Zenju brought this change] agent.c: Fix Unicode builds on Windows (#417) File: agent.c Notes: Fixes unicode builds for Windows in Visual Studio 16.3.2. Credit: Zenju - [Hans Meier brought this change] openssl.c: Fix use-after-free crash in openssl backend without memory leak (#439) Files: openssl.c Notes: Fixes memory leaks and use after free AES EVP_CIPHER contexts when using OpenSSL 1.0.x. Credit: Hans Meier - [Romain Geissler @ Amadeus brought this change] Session.c: Fix undefined warning when mixing with LTO-enabled libcurl. (#449) File: Session.c Notes: With gcc 9, libssh2, libcurl and LTO enabled for all binaries I see this warning (error with -Werror): vssh/libssh2.c: In function ‘ssh_statemach_act’: /data/mwrep/rgeissler/ospack/ssh2/BUILD/libssh2-libssh2-03c7c4a/src/session.c:579:9: error: ‘seconds_to_next’ is used uninitialized in this function [-Werror=uninitialized] 579 | int seconds_to_next; | ^ lto1: all warnings being treated as errors Gcc normally issues -Wuninitialized when it is sure there is a problem, and -Wmaybe-uninitialized when it's not sure, but it's possible. Here the compiler seems to have find a real case where this could happen. I looked in your code and overall it seems you always check if the return code is non null, not often that it's below zero. I think we should do the same here. With this patch, gcc is fine. Credit: Romain-Geissler-1A - [Zenju brought this change] transport.c: Fix crash with delayed compression (#443) Files: transport.c Notes: Fixes crash with delayed compression option using Bitvise server. Contributor: Zenju - [Will Cosgrove brought this change] Update INSTALL_MAKE path to INSTALL_MAKE.md (#446) Included for #429 - [Will Cosgrove brought this change] Update INSTALL_CMAKE filename to INSTALL_CMAKE.md (#445) Fixing for #429 - [Wallace Souza brought this change] Rename INSTALL_CMAKE to INTALL_CMAKE.md (#429) Adding Markdown file extension in order to Github render the instructions properly Will Cosgrove (17 Dec 2019) - [Daniel Stenberg brought this change] include/libssh2.h: fix comment: the known host key uses 4 bits (#438) - [Zenju brought this change] ssh-ed25519: Support PKIX + calc pubkey from private (#416) Files: openssl.c/h Author: Zenju Notes: Adds support for PKIX key reading by fixing: _libssh2_pub_priv_keyfile() is missing the code to extract the ed25519 public key from a given private key _libssh2_ed25519_new_private_frommemory is only parsing the openssh key format but does not understand PKIX (as retrieved via PEM_read_bio_PrivateKey) GitHub (15 Oct 2019) - [Will Cosgrove brought this change] .travis.yml: Fix Chrome and 32 bit builds (#423) File: .travis.yml Notes: * Fix Chrome installing by using Travis build in directive * Update to use libgcrypt20-dev package to fix 32 bit builds based on comments found here: https://launchpad.net/ubuntu/xenial/i386/libgcrypt11-dev - [Will Cosgrove brought this change] packet.c: improved parsing in packet_x11_open (#410) Use new API to parse data in packet_x11_open() for better bounds checking. Will Cosgrove (12 Sep 2019) - [Michael Buckley brought this change] knownhost.c: Double the static buffer size when reading and writing known hosts (#409) Notes: We had a user who was being repeatedly prompted to accept a server key repeatedly. It turns out the base64-encoded key was larger than the static buffers allocated to read and write known hosts. I doubled the size of these buffers. Credit: Michael Buckley GitHub (4 Sep 2019) - [Will Cosgrove brought this change] packet.c: improved packet parsing in packet_queue_listener (#404) * improved bounds checking in packet_queue_listener file: packet.c notes: improved parsing packet in packet_queue_listener - [Will Cosgrove brought this change] packet.c: improve message parsing (#402) * packet.c: improve parsing of packets file: packet.c notes: Use _libssh2_get_string API in SSH_MSG_DEBUG/SSH_MSG_DISCONNECT. Additional uint32 bounds check in SSH_MSG_GLOBAL_REQUEST. - [Will Cosgrove brought this change] misc.c: _libssh2_ntohu32 cast bit shifting (#401) To quite overly aggressive analyzers. Note, the builds pass, Travis is having some issues with Docker images. - [Will Cosgrove brought this change] kex.c: improve bounds checking in kex_agree_methods() (#399) file: kex.c notes: use _libssh2_get_string instead of kex_string_pair which does additional checks Will Cosgrove (23 Aug 2019) - [Fabrice Fontaine brought this change] acinclude.m4: add mbedtls to LIBS (#371) Notes: This is useful for static builds so that the Libs.private field in libssh2.pc contains correct info for the benefit of pkg-config users. Static link with libssh2 requires this information. Signed-off-by: Baruch Siach [Retrieved from: https://git.buildroot.net/buildroot/tree/package/libssh2/0002-acinclude.m4-add-mbedtls-to-LIBS.patch] Signed-off-by: Fabrice Fontaine Credit: Fabrice Fontaine - [jethrogb brought this change] Generate debug info when building with MSVC (#178) files: CMakeLists.txt notes: Generate debug info when building with MSVC credit: jethrogb - [Panos brought this change] Add agent forwarding implementation (#219) files: channel.c, test_agent_forward_succeeds.c, libssh2_priv.h, libssh2.h, ssh2_agent_forwarding.c notes: * Adding SSH agent forwarding. * Fix agent forwarding message, updated example. Added integration test code and cmake target. Added example to cmake list. credit: pkittenis GitHub (2 Aug 2019) - [Will Cosgrove brought this change] Update EditorConfig Added max_line_length = 80 - [Will Cosgrove brought this change] global.c : fixed call to libssh2_crypto_exit #394 (#396) * global.c : fixed call to libssh2_crypto_exit #394 File: global.c Notes: Don't call `libssh2_crypto_exit()` until `_libssh2_initialized` count is down to zero. Credit: seba30 Will Cosgrove (30 Jul 2019) - [hlefebvre brought this change] misc.c : Add an EWOULDBLOCK check for better portability (#172) File: misc.c Notes: Added support for all OS' that implement EWOULDBLOCK, not only VMS Credit: hlefebvre - [Etienne Samson brought this change] userauth.c: fix off by one error when loading public keys with no id (#386) File: userauth.c Credit: Etienne Samson Notes: Caught by ASAN: ================================================================= ==73797==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60700001bcf0 at pc 0x00010026198d bp 0x7ffeefbfed30 sp 0x7ffeefbfe4d8 READ of size 69 at 0x60700001bcf0 thread T0 2019-07-04 08:35:30.292502+0200 atos[73890:2639175] examining /Users/USER/*/libssh2_clar [73797] #0 0x10026198c in wrap_memchr (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x1f98c) #1 0x1000f8e66 in file_read_publickey userauth.c:633 #2 0x1000f2dc9 in userauth_publickey_fromfile userauth.c:1513 #3 0x1000f2948 in libssh2_userauth_publickey_fromfile_ex userauth.c:1590 #4 0x10000e254 in test_userauth_publickey__ed25519_auth_ok publickey.c:69 #5 0x1000090c3 in clar_run_test clar.c:260 #6 0x1000038f3 in clar_run_suite clar.c:343 #7 0x100003272 in clar_test_run clar.c:522 #8 0x10000c3cc in main runner.c:60 #9 0x7fff5b43b3d4 in start (libdyld.dylib:x86_64+0x163d4) 0x60700001bcf0 is located 0 bytes to the right of 80-byte region [0x60700001bca0,0x60700001bcf0) allocated by thread T0 here: #0 0x10029e053 in wrap_malloc (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5c053) #1 0x1000b4978 in libssh2_default_alloc session.c:67 #2 0x1000f8aba in file_read_publickey userauth.c:597 #3 0x1000f2dc9 in userauth_publickey_fromfile userauth.c:1513 #4 0x1000f2948 in libssh2_userauth_publickey_fromfile_ex userauth.c:1590 #5 0x10000e254 in test_userauth_publickey__ed25519_auth_ok publickey.c:69 #6 0x1000090c3 in clar_run_test clar.c:260 #7 0x1000038f3 in clar_run_suite clar.c:343 #8 0x100003272 in clar_test_run clar.c:522 #9 0x10000c3cc in main runner.c:60 #10 0x7fff5b43b3d4 in start (libdyld.dylib:x86_64+0x163d4) SUMMARY: AddressSanitizer: heap-buffer-overflow (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x1f98c) in wrap_memchr Shadow bytes around the buggy address: 0x1c0e00003740: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd 0x1c0e00003750: fd fd fd fd fd fd fd fa fa fa fa fa 00 00 00 00 0x1c0e00003760: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 0x1c0e00003770: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd 0x1c0e00003780: fd fd fa fa fa fa fd fd fd fd fd fd fd fd fd fa =>0x1c0e00003790: fa fa fa fa 00 00 00 00 00 00 00 00 00 00[fa]fa 0x1c0e000037a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc - [Thilo Schulz brought this change] openssl.c : Fix use-after-free crash on reinitialization of openssl backend file : openssl.c notes : libssh2's openssl backend has a use-after-free condition if HAVE_OPAQUE_STRUCTS is defined and you call libssh2_init() again after prior initialisation/deinitialisation of libssh2 credit : Thilo Schulz - [axjowa brought this change] openssl.h : Use of ifdef where if should be used (#389) File : openssl.h Notes : LIBSSH2_ECDSA and LIBSSH2_ED25519 are always defined so the #ifdef checks would never be false. This change makes it possible to build libssh2 against OpenSSL built without EC support. Change-Id: I0a2f07c2d80178314dcb7d505d1295d19cf15afd Credit : axjowa - [Zenju brought this change] Agent.c : Preserve error info from agent_list_identities() (#374) Files : agent.c Notes : Currently the error details as returned by agent_transact_pageant() are overwritten by a generic "agent list id failed" message by int agent_list_identities(LIBSSH2_AGENT* agent). Credit : Zenju - [Who? Me?! brought this change] Channel.c: Make sure the error code is set in _libssh2_channel_open() (#381) File : Channel.c Notes : if _libssh2_channel_open() fails, set the error code. Credit : mark-i-m - [Orgad Shaneh brought this change] Kex.c, Remove unneeded call to strlen (#373) File : Kex.c Notes : Removed call to strlen Credit : Orgad Shaneh - [Pedro Monreal brought this change] Spelling corrections (#380) Files : libssh2.h, libssh2_sftp.h, bcrypt_pbkdf.c, mbedtls.c, sftp.c, ssh2.c Notes : * Fixed misspellings Credit : Pedro Monreal - [Sebastián Katzer brought this change] Fix Potential typecast error for `_libssh2_ecdsa_key_get_curve_type` (#383) Issue : #383 Files : hostkey.c, crypto.h, openssl.c Notes : * Fix potential typecast error for `_libssh2_ecdsa_key_get_curve_type` * Rename _libssh2_ecdsa_key_get_curve_type to _libssh2_ecdsa_get_curve_type Credit : Sebastián Katzer GitHub (20 Jun 2019) - [Will Cosgrove brought this change] bump copyright date Version 1.9.0 (19 Jun 2019) GitHub (19 Jun 2019) - [Will Cosgrove brought this change] 1.9 Formatting - [Will Cosgrove brought this change] 1.9 Release notes Will Cosgrove (17 May 2019) - [Alexander Curtiss brought this change] libgcrypt.c : Fixed _libssh2_rsa_sha1_sign memory leak. (#370) File: libgcrypt.c Notes : Added calls to gcry_sexp_release to free memory allocated by gcry_sexp_find_token Credit : Reporter : beckmi PR by: Alexander Curtiss - [Orivej Desh brought this change] libssh2_priv.h : Fix musl build warning on sys/poll.h (#346) File : libssh2_priv.h Notes : musl prints `redirecting incorrect #include to ` http://git.musl-libc.org/cgit/musl/commit/include/sys/poll.h?id=54446d730cfb17c5f7bcf57f139458678f5066cc poll is defined by POSIX to be in poll.h: http://pubs.opengroup.org/onlinepubs/7908799/xsh/poll.html Credit : Orivej Desh GitHub (1 May 2019) - [Will Cosgrove brought this change] kex.c : additional bounds checks in diffie_hellman_sha1/256 (#361) Files : kex.c, misc.c, misc.h Notes : Fixed possible out of bounds memory access when reading malformed data in diffie_hellman_sha1() and diffie_hellman_sha256(). Added _libssh2_copy_string() to misc.c to return an allocated and filled char buffer from a string_buf offset. Removed no longer needed s var in kmdhgGPshakex_state_t. Will Cosgrove (26 Apr 2019) - [Tseng Jun brought this change] sftp.c : sftp_bin2attr() Correct attrs->gid assignment (#366) Regression with fix for #339 Credit : Tseng Jun - [Tseng Jun brought this change] kex.c : Correct type cast in curve25519_sha256() (#365) GitHub (24 Apr 2019) - [Will Cosgrove brought this change] transport.c : scope local total_num var (#364) file : transport.c notes : move local `total_num` variable inside of if block to prevent scope access issues which caused #360. Will Cosgrove (24 Apr 2019) - [doublex brought this change] transport.c : fixes bounds check if partial packet is read Files : transport.c Issue : #360 Notes : 'p->total_num' instead of local value total_num when doing bounds check. Credit : Doublex GitHub (23 Apr 2019) - [Will Cosgrove brought this change] Editor config file for source files (#322) Simple start to an editor config file when editing source files to make sure they are configured correctly. - [Will Cosgrove brought this change] misc.c : String buffer API improvements (#332) Files : misc.c, hostkey.c, kex.c, misc.h, openssl.c, sftp.c Notes : * updated _libssh2_get_bignum_bytes and _libssh2_get_string. Now pass in length as an argument instead of returning it to keep signedness correct. Now returns -1 for failure, 0 for success. _libssh2_check_length now returns 0 on success and -1 on failure to match the other string_buf functions. Added comment to _libssh2_check_length. Credit : Will Cosgrove Will Cosgrove (19 Apr 2019) - [doublex brought this change] mbedtls.c : _libssh2_mbedtls_rsa_new_private_frommemory() allow private-key from memory (#359) File : mbedtls.c Notes: _libssh2_mbedtls_rsa_new_private_frommemory() fixes private-key from memory reading to by adding NULL terminator before parsing; adds passphrase support. Credit: doublex - [Ryan Kelley brought this change] Session.c : banner_receive() from leaking when accessing non ssh ports (#356) File : session.c Release previous banner in banner_receive() if the session is reused after a failed connection. Credit : Ryan Kelley GitHub (11 Apr 2019) - [Will Cosgrove brought this change] Formatting in agent.c Removed whitespace. - [Will Cosgrove brought this change] Fixed formatting in agent.c Quiet linter around a couple if blocks and pointer. Will Cosgrove (11 Apr 2019) - [Zhen-Huan HWANG brought this change] sftp.c : discard and reset oversized packet in sftp_packet_read() (#269) file : sftp.c notes : when sftp_packet_read() encounters an sftp packet which exceeds SFTP max packet size it now resets the reading state so it can continue reading. credit : Zhen-Huan HWANG GitHub (11 Apr 2019) - [Will Cosgrove brought this change] Add agent functions libssh2_agent_get_identity_path() and libssh2_agent_set_identity_path() (#308) File : agent.c Notes : Libssh2 uses the SSH_AUTH_SOCK env variable to read the system agent location. However, when using a custom agent path you have to set this value using setenv which is not thread-safe. The new functions allow for a way to set a custom agent socket path in a thread safe manor. - [Will Cosgrove brought this change] Simplified _libssh2_check_length (#350) * Simplified _libssh2_check_length misc.c : _libssh2_check_length() Removed cast and improved bounds checking and format. Credit : Yuriy M. Kaminskiy - [Will Cosgrove brought this change] _libssh2_check_length() : additional bounds check (#348) Misc.c : _libssh2_check_length() Ensure the requested length is less than the total length before doing the additional bounds check Daniel Stenberg (25 Mar 2019) - misc: remove 'offset' from string_buf It isn't necessary. Closes #343 - sftp: repair mtime from e1ead35e475 A regression from e1ead35e4759 broke the SFTP mtime logic in sftp_bin2attr Also simplified the _libssh2_get_u32/u64 functions slightly. Closes #342 - session_disconnect: don't zero state, just clear the right bit If we clear the entire field, the freeing of data in session_free() is skipped. Instead just clear the bit that risk making the code get stuck in the transport functions. Regression from 4d66f6762ca3fc45d9. Reported-by: dimmaq on github Fixes #338 Closes #340 - libssh2_sftp.h: restore broken ABI Commit 41fbd44 changed variable sizes/types in a public struct which broke the ABI, which breaks applications! This reverts that change. Closes #339 - style: make includes and examples code style strict make travis and the makefile rule verify them too Closes #334 GitHub (21 Mar 2019) - [Daniel Stenberg brought this change] create a github issue template Daniel Stenberg (21 Mar 2019) - stale-bot: activated The stale bot will automatically mark stale issues (inactive for 90 days) and if still untouched after 21 more days, close them. See https://probot.github.io/apps/stale/ - libssh2_session_supported_algs.3: fix formatting mistakes Reported-by: Max Horn Fixes #57 - [Zenju brought this change] libssh2.h: Fix Error C2371 'ssize_t': redefinition Closes #331 - travis: add code style check Closes #324 - code style: unify code style Indent-level: 4 Max columns: 79 No spaces after if/for/while Unified brace positions Unified white spaces - src/checksrc.pl: code style checker imported as-is from curl Will Cosgrove (19 Mar 2019) - Merge branch 'MichaelBuckley-michaelbuckley-security-fixes' - Silence unused var warnings (#329) Silence warnings about unused variables in this test - Removed unneeded > 0 check When checking `userauth_kybd_num_prompts > 100` we don't care if it's also above zero. - [Matthew D. Fuller brought this change] Spell OpenSS_H_ right when talking about their specific private key (#321) Good catch, thanks. GitHub (19 Mar 2019) - [Will Cosgrove brought this change] Silence unused var warnings (#329) Silence warnings about unused variables in this test Michael Buckley (19 Mar 2019) - Fix more scope and printf warning errors - Silence unused variable warning GitHub (19 Mar 2019) - [Will Cosgrove brought this change] Removed unneeded > 0 check When checking `userauth_kybd_num_prompts > 100` we don't care if it's also above zero. Will Cosgrove (19 Mar 2019) - [Matthew D. Fuller brought this change] Spell OpenSS_H_ right when talking about their specific private key (#321) Good catch, thanks. Michael Buckley (18 Mar 2019) - Fix errors identified by the build process - Fix casting errors after merge GitHub (18 Mar 2019) - [Michael Buckley brought this change] Merge branch 'master' into michaelbuckley-security-fixes Michael Buckley (18 Mar 2019) - Move fallback SIZE_MAX and UINT_MAX to libssh2_priv.h - Fix type and logic issues with _libssh2_get_u64 Daniel Stenberg (17 Mar 2019) - examples: fix various compiler warnings - lib: fix various compiler warnings - session: ignore pedantic warnings for funcpointer <=> void * - travis: add a build using configure Closes #320 - configure: provide --enable-werror - appveyor: remove old builds that mostly cause failures ... and only run on master branch. Closes #323 - cmake: add two missing man pages to get installed too Both libssh2_session_handshake.3 and libssh2_userauth_publickey_frommemory.3 were installed by the configure build already. Reported-by: Arfrever on github Fixes #278 - include/libssh2.h: warning: "_WIN64" is not defined, evaluates to 0 We don't use #if for defines that might not be defined. - pem: //-comments are not allowed Will Cosgrove (14 Mar 2019) - [Daniel Stenberg brought this change] userauth: fix "Function call argument is an uninitialized value" (#318) Detected by scan-build. - fixed unsigned/signed issue Daniel Stenberg (15 Mar 2019) - session_disconnect: clear state If authentication is started but not completed before the application gives up and instead wants to shut down the session, the '->state' field might still be set and thus effectively dead-lock session_disconnect. This happens because both _libssh2_transport_send() and _libssh2_transport_read() refuse to do anything as long as state is set without the LIBSSH2_STATE_KEX_ACTIVE bit. Reported in curl bug https://github.com/curl/curl/issues/3650 Closes #310 Will Cosgrove (14 Mar 2019) - Release notes from 1.8.1 Michael Buckley (14 Mar 2019) - Use string_buf in sftp_init(). - Guard against out-of-bounds reads in publickey.c - Guard against out-of-bounds reads in session.c - Guard against out-of-bounds reads in userauth.c - Use LIBSSH2_ERROR_BUFFER_TOO_SMALL instead of LIBSSH2_ERROR_OUT_OF_BOUNDARY in sftp.c - Additional bounds checking in sftp.c - Additional length checks to prevent out-of-bounds reads and writes in _libssh2_packet_add(). https://libssh2.org/CVE-2019-3862.html - Add a required_size parameter to sftp_packet_require et. al. to require callers of these functions to handle packets that are too short. https://libssh2.org/CVE-2019-3860.html - Check the length of data passed to sftp_packet_add() to prevent out-of-bounds reads. - Prevent zero-byte allocation in sftp_packet_read() which could lead to an out-of-bounds read. https://libssh2.org/CVE-2019-3858.html - Sanitize padding_length - _libssh2_transport_read(). https://libssh2.org/CVE-2019-3861.html This prevents an underflow resulting in a potential out-of-bounds read if a server sends a too-large padding_length, possibly with malicious intent. - Defend against writing beyond the end of the payload in _libssh2_transport_read(). - Defend against possible integer overflows in comp_method_zlib_decomp. GitHub (14 Mar 2019) - [Will Cosgrove brought this change] Security fixes (#315) * Bounds checks Fixes for CVEs https://www.libssh2.org/CVE-2019-3863.html https://www.libssh2.org/CVE-2019-3856.html * Packet length bounds check CVE https://www.libssh2.org/CVE-2019-3855.html * Response length check CVE https://www.libssh2.org/CVE-2019-3859.html * Bounds check CVE https://www.libssh2.org/CVE-2019-3857.html * Bounds checking CVE https://www.libssh2.org/CVE-2019-3859.html and additional data validation * Check bounds before reading into buffers * Bounds checking CVE https://www.libssh2.org/CVE-2019-3859.html * declare SIZE_MAX and UINT_MAX if needed - [Will Cosgrove brought this change] fixed type warnings (#309) - [Will Cosgrove brought this change] Bumping version number for pending 1.8.1 release Will Cosgrove (4 Mar 2019) - [Daniel Stenberg brought this change] _libssh2_string_buf_free: use correct free (#304) Use LIBSSH2_FREE() here, not free(). We allow memory function replacements so free() is rarely the right choice... GitHub (26 Feb 2019) - [Will Cosgrove brought this change] Fix for building against libreSSL #302 Changed to use the check we use elsewhere. - [Will Cosgrove brought this change] Fix for when building against LibreSSL #302 Will Cosgrove (25 Feb 2019) - [gartens brought this change] docs: update libssh2_hostkey_hash.3 [ci skip] (#301) GitHub (21 Feb 2019) - [Will Cosgrove brought this change] fix malloc/free mismatches #296 (#297) - [Will Cosgrove brought this change] Replaced malloc with calloc #295 - [Will Cosgrove brought this change] Abstracted OpenSSL calls out of hostkey.c (#294) - [Will Cosgrove brought this change] Fix memory dealloc impedance mis-match #292 (#293) When using ed25519 host keys and a custom memory allocator. - [Will Cosgrove brought this change] Added call to OpenSSL_add_all_digests() #288 For OpenSSL 1.0.x we need to call OpenSSL_add_all_digests(). Will Cosgrove (12 Feb 2019) - [Zhen-Huan HWANG brought this change] SFTP: increase maximum packet size to 256K (#268) to match implementations like OpenSSH. - [Zenju brought this change] Fix https://github.com/libssh2/libssh2/pull/271 (#284) GitHub (16 Jan 2019) - [Will Cosgrove brought this change] Agent NULL check in shutdown #281 Will Cosgrove (15 Jan 2019) - [Adrian Moran brought this change] mbedtls: Fix leak of 12 bytes by each key exchange. (#280) Correctly free ducts by calling _libssh2_mbedtls_bignum_free() in dtor. - [alex-weaver brought this change] Fix error compiling on Win32 with STDCALL=ON (#275) GitHub (8 Nov 2018) - [Will Cosgrove brought this change] Allow default permissions to be used in sftp_mkdir (#271) Added constant LIBSSH2_SFTP_DEFAULT_MODE to use the server default permissions when making a new directory Will Cosgrove (13 Sep 2018) - [Giulio Benetti brought this change] openssl: fix dereferencing ambiguity potentially causing build failure (#267) When dereferencing from *aes_ctr_cipher, being a pointer itself, ambiguity can occur; fixed possible build errors. Viktor Szakats (12 Sep 2018) - win32/GNUmakefile: define HAVE_WINDOWS_H This macro was only used in test/example code before, now it is also used in library code, but only defined automatically by automake/cmake, so let's do the same for the standalone win32 make file. It'd be probably better to just rely on the built-in _WIN32 macro to detect the presence of windows.h though. It's already used in most of libssh2 library code. There is a 3rd, similar macro named LIBSSH2_WIN32, which might also be replaced with _WIN32. Ref: https://github.com/libssh2/libssh2/commit/8b870ad771cbd9cd29edbb3dbb0878e950f868ab Closes https://github.com/libssh2/libssh2/pull/266 Marc Hoersken (2 Sep 2018) - Fix conditional check for HAVE_DECL_SECUREZEROMEMORY "Unlike the other `AC_CHECK_*S' macros, when a symbol is not declared, HAVE_DECL_symbol is defined to `0' instead of leaving HAVE_DECL_symbol undeclared. When you are sure that the check was performed, use HAVE_DECL_symbol in #if." Source: autoconf documentation for AC_CHECK_DECLS. - Fix implicit declaration of function 'SecureZeroMemory' Include window.h in order to use SecureZeroMemory on Windows. - Fix implicit declaration of function 'free' by including stdlib.h GitHub (27 Aug 2018) - [Will Cosgrove brought this change] Use malloc abstraction function in pem parse Fix warning on WinCNG build. - [Will Cosgrove brought this change] Fixed possible junk memory read in sftp_stat #258 - [Will Cosgrove brought this change] removed INT64_C define (#260) No longer used. - [Will Cosgrove brought this change] Added conditional around engine.h include Will Cosgrove (6 Aug 2018) - [Alex Crichton brought this change] Fix OpenSSL link error with `no-engine` support (#259) This commit fixes linking against an OpenSSL library that was compiled with `no-engine` support by bypassing the initialization routines as they won't be available anyway. GitHub (2 Aug 2018) - [Will Cosgrove brought this change] ED25519 Key Support #39 (#248) OpenSSH Key and ED25519 support #39 Added _libssh2_explicit_zero() to explicitly zero sensitive data in memory #120 * ED25519 Key file support - Requires OpenSSL 1.1.1 or later * OpenSSH Key format reading support - Supports RSA/DSA/ECDSA/ED25519 types * New string buffer reading functions - These add build-in bounds checking and convenance methods. Used for OpenSSL PEM file reading. * Added new tests for OpenSSH formatted Keys - [Will Cosgrove brought this change] ECDSA key types are now explicit (#251) * ECDSA key types are now explicit Issue was brough up in pull request #248 Will Cosgrove (2 May 2018) - [Jakob Egger brought this change] Add Instructions for building from Master (#249) GitHub (27 Apr 2018) - [Will Cosgrove brought this change] Initialize sb_intl #226 Will Cosgrove (19 Apr 2018) - [doublex brought this change] buffer overflow (valgrind) (#159) - [Brendan Shanks brought this change] mbedTLS: Remove some C99-style intermingled variable declarations (#196) GitHub (18 Apr 2018) - [Will Cosgrove brought this change] fix for #160 Will Cosgrove (18 Apr 2018) - [doublex brought this change] fix memory leak when using mbedtls backend (#158) _libssh2_bn_init_from_bin/_libssh2_bn_free would leak bignum from mbedtls_calloc(). - [Brendan Shanks brought this change] mbedTLS: Avoid multiple definition errors for context handles (#197) - [Tseng Jun brought this change] Fix the EVP cipher meth memory leakage problem (#244) * Fix the EVP cipher meth memory leakage problem Looks good, thanks for the fixes. Marc Hörsken (31 Mar 2018) - [Will Cosgrove brought this change] Added ECDSA defines for WinCNG (#245) Fixed missing defines preventing building using WinCNG GitHub (30 Mar 2018) - [Will Cosgrove brought this change] Fix for _libssh2_rsa_new with OpenSSL 1.0.x missing d value assignment. Will Cosgrove (20 Mar 2018) - [Etienne Samson brought this change] A collection of small fixes (#198) * tests: Remove if-pyramids * tests: Switch run_command arguments * tests: Make run_command a vararg function * tests: Xcode doesn't obey CMake's test working directory * openssl: move manual AES-CTR cipher into crypto init * cmake: Move our include dir before all other include paths GitHub (15 Mar 2018) - [Will Cosgrove brought this change] Fixes incorrect indexing of KEX prefs string After stripping out an invalid KEX pref entry, it would incorrectly advance again leaving invalid values in the list. Viktor Szakats (13 Mar 2018) - tests: fix checksrc warnings Also: * add 'static' qualifier to file-wide const buffers * fix a non-ANSI C89 comment * silence a mismatched fprintf() mask warning by adding a cast - cmake: recognize OpenSSL 1.1 .dll names Also fix some comment typos and a stray tab. - docs: update an URL [ci skip] Daniel Stenberg (12 Mar 2018) - docs/SECURITY: the max embargo is 14 days now Viktor Szakats (12 Mar 2018) - docs: spelling fixes [ci skip] Closes https://github.com/libssh2/libssh2/pull/222 GitHub (12 Mar 2018) - [Will Cosgrove brought this change] Fixed minor tabs/spacing issues - [Will Cosgrove brought this change] Update kex.c - [Will Cosgrove brought this change] Added basic bounds checking #206 Basic bounds checking in ecdh_sha2_nistp() - [Will Cosgrove brought this change] Fixed Clang warning #206 Fixed possible garbage value for secret in an error case - [Will Cosgrove brought this change] Fixed incorrect #if to #ifdef #206 When checking HAVE_OPAQUE_STRUCTS. Viktor Szakats (12 Mar 2018) - src: suppress two checksrc warnings Ref: https://github.com/libssh2/libssh2/pull/235 - src: address fopen() warnings, add missing copyright headers Ref: https://github.com/libssh2/libssh2/pull/235 - src: replace sprintf() with snprintf() Ref: https://github.com/libssh2/libssh2/pull/235 - src: fix checksrc warnings Use checksrc.pl from the curl project, with (for now) suppressed long line warnings and indentation set to 4 spaces. Fixes are whitespace for the most part. Warning count went down from 2704 to 12. Also fix codespell typos, two non-ANSI C89 comments and a stray tab in include/libssh2.h. Ref: https://github.com/libssh2/libssh2/pull/235 - checksrc: add source style checker This is a slightly extended version of this original source from the curl project: https://github.com/curl/curl/blob/8b754c430b9a4c51aa606c687ee5014faf7c7b06/lib/checksrc.pl This version adds the following options to customize it for libssh2 (plus some whitespace formatting): `-i` to override indentation spaces (2) `-m` to override maximum line length (79) Command-line used to check libssh2 sources: $ ./checksrc.pl -i4 -m500 *.c *.h Closes https://github.com/libssh2/libssh2/pull/236 - src: add static qualifier To private, const strings. Closes https://github.com/libssh2/libssh2/pull/237 - [Will Cosgrove brought this change] Add support for ECDSA keys and host keys (#41) This commit lands full ECDSA key support when using the OpenSSL backend. Which includes: New KEX methods: ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521 Can now read OpenSSL formatted ECDSA key files. Now supports known host keys of type ecdsa-sha2-nistp256. New curve types: NID_X9_62_prime256v1, NID_secp384r1, NID_secp521r1 Default host key preferred ordering is now nistp256, nistp384, nistp521, rsa, dss. Ref: https://github.com/libssh2/libssh2/issues/41 Closes https://github.com/libssh2/libssh2/pull/206 GitHub (15 Dec 2017) - [Will Cosgrove brought this change] Fixed possible crash when decoding invalid data When trying to decode invalid data, it frees the buffer but doesn't nil it so the caller gets a junk memory pointer which they could potentially double free. - [Will Cosgrove brought this change] Remove call to OpenSSL_add_all_ciphers() Now lives in libssh2 init() from PR #189 - [Will Cosgrove brought this change] Fixed incorrect reference to decrypted block Fixed incorrectly copied memory from p->buf into init instead of from the decrypted buffer block. The only reason this worked was because the crypt() function decrypts the value in-place and overwrites p->buf. I'm working on a fork that no longer does this and exposed this bug. Will Cosgrove (20 Oct 2017) - [Pan brought this change] Fix typo in crypt.c (#218) Kamil Dudka (17 Oct 2017) - session: avoid printing misleading debug messages ... while throwing LIBSSH2_ERROR_EAGAIN out of session_startup() If the session runs in blocking mode, LIBSSH2_ERROR_EAGAIN never reaches the libssh2 API boundary and, in non-blocking mode, these messages are suppressed by the condition in _libssh2_error_flags() anyway. Closes #211 Viktor Szakats (15 Oct 2017) - win32/GNUmakefile: allow customizing dll suffixes - New `LIBSSH2_DLL_SUFFIX` envvar will add a suffix to the generated libssh2 dll name. Useful to add `-x64` to 64-bit builds so that it can live in the same directory as the 32-bit one. By default this is empty. - New `LIBSSH2_DLL_A_SUFFIX` envvar to customize the suffix of the generated import library (implib) for libssh2 .dll. It defaults to `dll`, and it's useful to modify that to `.dll` to have the standard naming scheme for mingw-built .dlls, i.e. `libssh2.dll.a`. Ref: https://github.com/curl/curl/commit/aaa16f80256abc1463fd9374815130a165222257 Closes https://github.com/libssh2/libssh2/pull/215 - makefile.m32: allow to override gcc, ar and ranlib Allow to ovverride certain build tools, making it possible to use LLVM/Clang to build libssh2. The default behavior is unchanged. To build with clang (as offered by MSYS2), these settings can be used: LIBSSH2_CC=clang LIBSSH2_AR=llvm-ar LIBSSH2_RANLIB=llvm-ranlib Also adjust ranlib parameters to be compatible with LLVM/Clang's ranlib tool. Closes https://github.com/libssh2/libssh2/pull/214 GitHub (27 Sep 2017) - [Will Cosgrove brought this change] Fixes out of bounds memory access (#210) If an invalid PEM file is read and the lines are longer than 128 characters it will go out of bounds and crash on line 91. Will Cosgrove (11 Sep 2017) - [Kamil Dudka brought this change] scp: do not NUL-terminate the command for remote exec (#208) It breaks SCP download/upload from/to certain server implementations. The bug does not manifest with OpenSSH, which silently drops the NUL byte (eventually with any garbage that follows the NUL byte) before executing it. Bug: https://bugzilla.redhat.com/1489736 GitHub (21 Aug 2017) - [Viktor Szakats brought this change] openssl.c: remove no longer used variable (#204) after e378d2e30a40bd9bcee06dc3a4250f269098e200 - [Will Cosgrove brought this change] Fix for #188 (#189) * Update openssl.c * Create openssl.h Will Cosgrove (24 May 2017) - [Marcel Raad brought this change] openssl: fix build with OpenSSL 1.1 API (#176) When building with OPENSSL_API_COMPAT=0x10100000L, OpenSSL_add_all_algorithms and OpenSSL_add_all_ciphers don't exist. The corresponding functionality is handled automatically with OpenSSL 1.1. - [Sune Bredahl brought this change] Add support for SHA256 hostkey fingerprints (#180) Looks good, thanks! GitHub (12 May 2017) - [Will Cosgrove brought this change] Fix memory leak of crypt_ctx->h using openSSL 1.1+ (#177) Need to use EVP_CIPHER_CTX_free instead of EVP_CIPHER_CTX_reset. Marc Hoersken (2 Mar 2017) - tests/openssh_server/authorized_keys: add key_rsa_encrypted.pub - tests: add simple test for passphrase-protected PEM file support - os400qc3: enable passphrase-protected PEM file support using pem.c - pem: fix indentation and replace assert after 386e012292 - [Keno Fischer brought this change] pem: add passphrase-protected PEM file support for libgcrypt and wincng Since they use our own PEM parser which did not support encrypted PEM files, trying to use such files on these backends failed. Fix that by augmenting the PEM parser to support encrypted PEM files. - [Thomas brought this change] misc: use time constant implementation for AES CTR increment - [Thomas brought this change] wincng: add AES CTR mode support (aes128-ctr, aes192-ctr, aes256-ctr) - [Thomas brought this change] openssl: move shared AES-CTR code into misc Daniel Stenberg (20 Dec 2016) - [Alex Crichton brought this change] kex: acknowledge error code from libssh2_dh_key_pair() Fixes a segfault using ssh-agent on Windows This commit fixes a segfault seen dereferencing a null pointer on Windows when using ssh-agent. The problem ended up being that errors weren't being communicated all the way through, causing null pointers to be used when functions should have bailed out sooner. The `_libssh2_dh_key_pair` function for WinCNG was modified to propagate errors, and then the two callsites in kex.c of `diffie_hellman_sha{1,256}` were updated to recognize this error and bail out. Fixes #162 Closes #163 Alexander Lamaison (27 Nov 2016) - [monnerat brought this change] Implement Diffie-Hellman computations in crypto backends. (#149) Not all backends feature the low level API needed to compute a Diffie-Hellman secret, but some of them directly implement Diffie-Hellman support with opaque private data. The later approach is now generalized and backends are responsible for all Diffie Hellman computations. As a side effect, procedures/macros _libssh2_bn_rand and _libssh2_bn_mod_exp are no longer needed outside the backends. Peter Stuge (16 Nov 2016) - acinclude.m4: The mbedtls crypto backend actually requires libmbedcrypto Examples can't be linked with libmbedtls but need libmbedcrypto, and any users of libssh2 which use libtool and libssh2.la would encounter the same problem. This changes the mbedtls detection to search for libmbedcrypto, which is the actual dependency for the backend. - acinclude.m4: Add CPPFLAGS=-I$prefix-dir/include in LIBSSH2_LIB_HAVE_LINKFLAGS This is absolutely neccessary for header files to be found when AC_LIB_HAVE_LINKFLAGS searches for libraries. - acinclude.m4: Make saved variables in LIBSSH2_LIB_HAVE_LINKFLAGS uniform - docs/HACKING.CRYPTO: Improve documentation for autoconf build system Alexander Lamaison (16 Nov 2016) - [Alex Arslan brought this change] Check for netinet/in.h in the tests cmake file (#148) - [Patrick Monnerat brought this change] Define new Diffie-Hellman context for mbedTLS - [monnerat brought this change] Make libssh2 work again on os400. (#118) * os400: minimum supported OS version is now V6R1. Do not log compiler informational messages. * Implement crypto backend specific Diffie-Hellman computation. This feature is now needed on os400 because the QC3 library does not implement bn_mod_exp() natively. Up to now, this function was emulated using an RSA encryption, but commits ca5222ea819cc5ed797860070b4c6c1aeeb28420 and 7934c9ce2a029c43e3642a492d3b9e494d1542be (CVE-2016-0787) broke the emulation because QC3 only supports RSA exponents up to 512 bits. Happily, QC3 supports a native API for Diffie-Hellman computation, with opaque random value: this commit implements the use of this API and, as a side effect, enables support of this feature for any other crypto backend that would use it. A "generic" Diffie-Hellman computation internal API supports crypto backends not implementing their own: this generic API uses the same functions as before. * Fix typos in docs/HACKING.CRYPTO. - [Peter Stuge brought this change] acinclude.m4: Fixup OpenSSL EVP_aes_128_ctr() detection - [Peter Stuge brought this change] configure.ac: Add --with-crypto= instead of many different --with-$backend The new --with-crypto option replaces the previous backend-specific --with-{openssl,libgcrypt,mbedtls,wincng} options and fixes some issues. * libgcrypt or mbedtls would previously be used whenever found, even if configure was passed --without-libgcrypt or --without-mbedtls. * If --with-$backend was specified then configure would not fail even if that library could not be found, and would instead use whichever crypto library was found first. The new option defaults to `auto`, which makes configure check for all supported crypto libraries in turn, choosing the first one found, or exiting with an error if none can be found. - [Tony Kelman brought this change] Build mbedtls from source on Travis (#133) * Revert "Revert "travis: Test mbedtls too"" This reverts commit c4c60eac5ca756333034b07dd9e0b97741493ed3. * travis: Build mbedtls from source on Travis Use TOOLCHAIN_OPTION when calling cmake on mbedtls * tests: only run DSA tests for non-mbedtls crypto backends - [Peter Stuge brought this change] configure.ac src/Makefile.am: Remove dead AM_CONDITIONAL(OS400QC3) According to os400/README400 this backend can not be built with configure+make, and the conditional is hard coded to false. - [Peter Stuge brought this change] configure.ac: Add -DNDEBUG to CPPFLAGS in non-debug builds There are a few uses of assert() in channel.c, sftp.c and transport.c. - [Peter Stuge brought this change] src/global.c: Fix conditional AES-CTR support Most of libssh2 already has conditional support for AES-CTR according to the LIBSSH2_AES_CTR crypto backend #define, but global.c needed fixing. - [Peter Stuge brought this change] src/crypto.h src/userauth.c: Fix conditional RSA support Most of libssh2 already has conditional support for RSA according to the LIBSSH2_RSA crypto backend #define, but crypto.h and userauth.c needed a few small fixes. - [Peter Stuge brought this change] src/kex.c: Cast libssh2_sha{1,256}_update data arguments properly The update functions take a const unsigned char * but were called with (const) char * in some places, causing unneccessary warnings. - [Peter Stuge brought this change] docs/HACKING.CRYPTO: Fix two type typos - [Sergei Trofimovich brought this change] acinclude.m4: fix ./configure --with-libgcrypt The change fixes passing of bogus gcrypt prefix. Reproducible as: $ ./configure --with-libgcrypt $ make V=1 ... /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -Iyes/include -version-info 1:1:0 -no-undefined -export-symbols-regex '^libssh2_.*' -lgcrypt -lz -Lyes/lib -o libssh2.la -rpath /usr/local/lib channel.lo comp.lo crypt.lo hostkey.lo kex.lo mac.lo misc.lo packet.lo publickey.lo scp.lo session.lo sftp.lo userauth.lo transport.lo version.lo knownhost.lo agent.lo libgcrypt.lo pem.lo keepalive.lo global.lo -lgcrypt ../libtool: line 7475: cd: yes/lib: No such file or directory libtool: error: cannot determine absolute directory name of 'yes/lib' These -Iyes/include -Lyes/lib come from libgcrypt code autodetection: if test -n "$use_libgcrypt" && test "$use_libgcrypt" != "no"; then LDFLAGS="$LDFLAGS -L$use_libgcrypt/lib" CFLAGS="$CFLAGS -I$use_libgcrypt/include" I assume it's a typo to use yes/no flag as a prefix and changed it to '$with_libgcrypt_prefix'. Reported-by: Mikhail Pukhlikov Signed-off-by: Sergei Trofimovich - [Zenju brought this change] libssh2_sftp_init hang: last error not set The problem is that the original if statement simply returns NULL, but does not set the session last error code. The consequence is that libssh2_sftp_init() also returns NULL and libssh2_session_last_errno(sshSession) == LIBSSH2_ERROR_NONE. In my test the LIBSSH2_ERROR_EAGAIN is coming from sftp.c row 337: if(4 != sftp->partial_size_len) /* we got a short read for the length part */ return LIBSSH2_ERROR_EAGAIN; with "partial_size_len == 0". Not sure if this is expected. - [Aidan Hobson Sayers brought this change] docs: correctly describe channel_wait_eof channel_wait_eof waits for channel->remote.eof, which is set on receiving a `SSH_MSG_CHANNEL_EOF` message. This message is sent when a party has no more data to send on a channel. - [Zenju brought this change] Fix MSVC 14 compilation warning (#92) 1> sftp.c 1>libssh2-files\src\sftp.c(3393): warning C4456: declaration of 'retcode' hides previous local declaration 1> libssh2-files\src\sftp.c(3315): note: see declaration of 'retcode' - [Salvador Fandino brought this change] LIBSSH2_ERROR_CHANNEL_WINDOW_FULL: add new error code In order to signal that the requested operation can not succeed because the receiving window had been exhausted, the error code LIBSSH2_ERROR_BUFFER_TOO_SMALL has been reused but I have found that in certain context it may be ambigous. This patch introduces a new error code, LIBSSH2_ERROR_CHANNEL_WINDOW_FULL, exclusive to signal that condition. - [Salvador Fandino brought this change] channel_wait_eof: handle receive window exhaustion Until now, in blocking mode, if the remote receiving window is exhausted this function hangs forever as data is not read and the remote side just keeps waiting for the window to grow before sending more data. This patch, makes this function check for that condition and abort with an error when it happens. - [Salvador Fandino brought this change] channel_wait_closed: don't fail when unread data is queued This function was calling channel_wait_eof to ensure that the EOF packet has already been received, but that function also checks that the read data queue is empty before reporting the EOF. That caused channel_wait_closed to fail with a LIBSSH2_ERROR_INVAL when some data was queued even after a successful call to libssh2_channel_wait_eof. This patch changes libssh2_channel_wait_closed to look directly into channel->remote.eof so that both libssh2_channel_wait_eof and libssh2_channel_wait_closed bahave consistently. - [Salvador Fandino brought this change] channel_wait_eof: fix debug message Daniel Stenberg (25 Oct 2016) - libssh2.h: start working on 1.8.1 Version 1.8.0 (25 Oct 2016) Daniel Stenberg (25 Oct 2016) - RELEASE-NOTES: adjusted for 1.8.0 Kamil Dudka (20 Oct 2016) - Revert "aes: the init function fails when OpenSSL has AES support" This partially reverts commit f4f2298ef3635acd031cc2ee0e71026cdcda5864 because it caused the compatibility code to call initialization routines redundantly, leading to memory leakage with OpenSSL 1.1 and broken curl test-suite in Fedora: 88 bytes in 1 blocks are definitely lost in loss record 5 of 8 at 0x4C2DB8D: malloc (vg_replace_malloc.c:299) by 0x72C607D: CRYPTO_zalloc (mem.c:100) by 0x72A2480: EVP_CIPHER_meth_new (cmeth_lib.c:18) by 0x4E5A550: make_ctr_evp.isra.0 (openssl.c:407) by 0x4E5A8E8: _libssh2_init_aes_ctr (openssl.c:471) by 0x4E5BB5A: libssh2_init (global.c:49) Daniel Stenberg (19 Oct 2016) - [Charles Collicutt brought this change] libssh2_wait_socket: Fix comparison with api_timeout to use milliseconds (#134) Fixes #74 - [Charles Collicutt brought this change] Set err_msg on _libssh2_wait_socket errors (#135) - Revert "travis: Test mbedtls too" This reverts commit 3e6de50a24815e72ec5597947f1831f6083b7da8. Travis doesn't seem to support the mbedtls-dev package - maketgz: support "only" to only update version number locally and fix the date output locale - configure: make the --with-* options override the OpenSSL default ... previously it would default to OpenSSL even with the --with-[crypto] options used unless you specificly disabled OpenSSL. Now, enabling another backend will automatically disable OpenSSL if the other one is found. - [Keno Fischer brought this change] docs: Add documentation on new cmake/configure options - [Keno Fischer brought this change] configure: Add support for building with mbedtls - [wildart brought this change] travis: Test mbedtls too - [wildart brought this change] crypto: add support for the mbedTLS backend Closes #132 - [wildart brought this change] cmake: Add CLEAR_MEMORY option, analogously to that for autoconf - README.md: fix link typo - README: markdown version to look nicer on github Viktor Szakats (5 Sep 2016) - [Taylor Holberton brought this change] openssl: add OpenSSL 1.1.0 compatibility Daniel Stenberg (4 Sep 2016) - [Antenore Gatta brought this change] tests: HAVE_NETINET_IN_H was not defined correctly (#127) Fixes #125 - SECURITY: fix web site typo - SECURITY: security process GitHub (14 Aug 2016) - [Alexander Lamaison brought this change] Basic dockerised test suite. This introduces a test suite for libssh2. It runs OpenSSH in a Docker container because that works well on Windows (via docker-machine) as well as Linux. Presumably it works on Mac too with docker-machine, but I've not tested that. Because the test suite is docker-machine aware, you can also run it against a cloud provider, for more realistic network testing, by setting your cloud provider as your active docker machine. The Appveyor CI setup in this commit does that because Appveyor doesn't support docker locally. Kamil Dudka (3 Aug 2016) - [Viktor Szakats brought this change] misc.c: Delete unused static variables Closes #114 Daniel Stenberg (9 Apr 2016) - [Will Cosgrove brought this change] Merge pull request #103 from willco007/patch-2 Fix for security issue CVE-2016-0787 Alexander Lamaison (2 Apr 2016) - [Zenju brought this change] Fix MSVC 14 compilation errors For _MSC_VER == 1900 these macros are not needed and create problems: 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\stdio.h(1925): warning C4005: 'snprintf': macro redefinition (compiling source file libssh2-files\src\mac.c) 1> \win32\libssh2_config.h(27): note: see previous definition of 'snprintf' (compiling source file libssh2-files\src\mac.c) 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\stdio.h(1927): fatal error C1189: #error: Macro definition of snprintf conflicts with Standard Library function declaration (compiling source file libssh2-files\src\mac.c) Daniel Stenberg (26 Mar 2016) - [Brad Harder brought this change] _libssh2_channel_open: speeling error fixed in channel error message Alexander Lamaison (15 Mar 2016) - Link with crypt32.lib on Windows. Makes linking with static OpenSSL work again. Although it's not required for dynamic OpenSSL, it does no harm. Fixes #98. - [Craig A. Berry brought this change] Tweak VMS help file building. Primarily this is handling cases where top-level files moved into the docs/ directory. I also corrected a typo and removed the claim that libssh2 is public domain. - [Craig A. Berry brought this change] Build with standard stat structure on VMS. This gets us large file support, is available on any VMS release in the last decade and more, and gives stat other modern features such as 64-bit ino_t. - [Craig A. Berry brought this change] Update vms/libssh2_config.h. VMS does have stdlib.h, gettimeofday(), and OpenSSL. The latter is appropriate to hard-wire in the configuration because it's installed by default as part of the base operating system and there is currently no libgcrypt port. - [Craig A. Berry brought this change] VMS can't use %zd for off_t format. %z is a C99-ism that VMS doesn't currently have; even though the compiler is C99-compliant, the library isn't quite. The off_t used for the st_size element of the stat can be 32-bit or 64-bit, so detect what we've got and pick a format accordingly. - [Craig A. Berry brought this change] Normalize line endings in libssh2_sftp_get_channel.3. Somehow it got Windows-style CRLF endings so convert to just LF, for consistency as well as not to confuse tools that will regard the \r as content (e.g. the OpenVMS help librarian). Dan Fandrich (29 Feb 2016) - libgcrypt: Fixed a NULL pointer dereference on OOM Daniel Stenberg (24 Feb 2016) - [Viktor Szakats brought this change] url updates, HTTP => HTTPS Closes #87 Dan Fandrich (23 Feb 2016) - RELEASE-NOTES: removed some duplicated names Version 1.7.0 (23 Feb 2016) Daniel Stenberg (23 Feb 2016) - web: the site is now HTTPS - RELEASE-NOTES: 1.7.0 release - diffie_hellman_sha256: convert bytes to bits As otherwise we get far too small numbers. Reported-by: Andreas Schneider CVE-2016-0787 Alexander Lamaison (18 Feb 2016) - Allow CI failures with VS 2008 x64. Appveyor doesn't support this combination. Daniel Stenberg (16 Feb 2016) - [Viktor Szakats brought this change] GNUmakefile: list system libs after user libs Otherwise some referenced WinSock functions will fail to resolve when linking against LibreSSL 2.3.x static libraries with mingw. Closes #80 - [Viktor Szakats brought this change] openssl: apply new HAVE_OPAQUE_STRUCTS macro Closes #81 - [Viktor Szakats brought this change] openssl: fix LibreSSL support after OpenSSL 1.1.0-pre1/2 support Alexander Lamaison (14 Feb 2016) - sftp.h: Fix non-C90 type. uint64_t does not exist in C90. Use libssh2_uint64_t instead. - Exclude sshd tests from AppVeyor. They fail complaining that sshd wasn't invoked with an absolute path. - Test on more versions of Visual Studio. - Fix Appveyor builds. Daniel Stenberg (14 Feb 2016) - [Viktor Szakats brought this change] openssl: add OpenSSL 1.1.0-pre3-dev compatibility by using API instead of accessing an internal structure. Closes #83 - RELEASE-NOTES: synced with 996b04ececdf - include/libssh2.h: next version is 1.7.0 - configure: build "silent" if possible - sftp: re-indented some minor stuff - [Jakob Egger brought this change] sftp.c: ensure minimum read packet size For optimum performance we need to ensure we don't request tiny packets. - [Jakob Egger brought this change] sftp.c: Explicit return values & sanity checks - [Jakob Egger brought this change] sftp.c: Check Read Packet File Offset This commit adds a simple check to see if the offset of the read request matches the expected file offset. We could try to recover, from this condition at some point in the future. Right now it is better to return an error instead of corrupted data. - [Jakob Egger brought this change] sftp.c: Don't return EAGAIN if data was written to buffer - [Jakob Egger brought this change] sftp.c: Send at least one read request before reading This commit ensures that we have sent at least one read request before we try to read data in sftp_read(). Otherwise sftp_read() would return 0 bytes (indicating EOF) if the socket is not ready for writing. - [Jakob Egger brought this change] sftp.c: stop reading when buffer is full Since we can only store data from a single chunk in filep, we have to stop receiving data as soon as the buffer is full. This adresses the following bug report: https://github.com/libssh2/libssh2/issues/50 Salvador Fandiño (21 Jan 2016) - agent_disconnect_unix: unset the agent fd after closing it "agent_disconnect_unix", called by "libssh2_agent_disconnect", was leaving the file descriptor in the agent structure unchanged. Later, "libssh2_agent_free" would call again "libssh2_agent_disconnect" under the hood and it would try to close again the same file descriptor. In most cases that resulted in just a harmless error, but it is also possible that the file descriptor had been reused between the two calls resulting in the closing of an unrelated file descriptor. This patch sets agent->fd to LIBSSH2_INVALID_SOCKET avoiding that issue. Signed-off-by: Salvador Fandiño Daniel Stenberg (18 Jan 2016) - [Patrick Monnerat brought this change] os400qc3: support encrypted private keys PKCS#8 EncryptedPrivateKeyinfo structures are recognized and decoded to get values accepted by the Qc3 crypto library. - [Patrick Monnerat brought this change] os400qc3: New PKCS#5 decoder The Qc3 library is not able to handle PKCS#8 EncryptedPrivateKeyInfo structures by itself. It is only capable of decrypting the (encrypted) PrivateKeyInfo part, providing a key encryption key and an encryption algorithm are given. Since the encryption key and algorithm description part in a PKCS#8 EncryptedPrivateKeyInfo is a PKCS#5 structure, such a decoder is needed to get the derived key method and hash, as well as encryption algorith and initialisation vector. - [Patrick Monnerat brought this change] os400qc3: force continuous update on non-final hash/hmac computation - [Patrick Monnerat brought this change] os400qc3: Be sure hmac keys have a minimum length The Qc3 library requires a minimum key length depending on the target hash algorithm. Append binary zeroes to the given key if not long enough. This matches RFC 2104 specifications. - [Patrick Monnerat brought this change] os400qc3: Slave descriptor for key encryption key The Qc3 library requires the key encryption key to exist as long as the encrypted key is used. Its descriptor token is then kept as an "encrypted key slave" for recursive release. - [Patrick Monnerat brought this change] os400qc3.c: comment PEM/DER decoding - [Patrick Monnerat brought this change] os400qc3.c: improve ASN.1 header byte checks - [Patrick Monnerat brought this change] os400qc3.c: improve OID matching - [Patrick Monnerat brought this change] os400: os400qc3.c: replace malloc by LIBSSH2_ALLOC or alloca where possible - [Patrick Monnerat brought this change] os400: asn1_new_from_bytes(): use data from a single element only - [Patrick Monnerat brought this change] os400: fix an ILE/RPG prototype - [Patrick Monnerat brought this change] os400: implement character encoding conversion support - [Patrick Monnerat brought this change] os400: do not miss some external prototypes Build procedure extproto() did not strip braces from header files, thus possibly prepended them to true prototypes. This prevented the prototype to be recognized as such. The solution implemented here is to map braces to semicolons, effectively considering them as potential prototype delimiters. - [Patrick Monnerat brought this change] os400: Really add specific README - [Patrick Monnerat brought this change] os400: Add specific README and include new files in dist tarball - [Patrick Monnerat brought this change] os400: add compilation scripts - [Patrick Monnerat brought this change] os400: include files for ILE/RPG In addition, file os400/macros.h declares all procedures originally defined as macros. It must not be used for real inclusion and is only intended to be used as a `database' for macro wrapping procedures generation. - [Patrick Monnerat brought this change] os400: add supplementary header files/wrappers. Define configuration. - [Patrick Monnerat brought this change] Protect callback function calls from macro substitution Some structure fields holding callback addresses have the same name as the underlying system function (connect, send, recv). Set parentheses around their reference to suppress a possible macro substitution. Use a macro for connect() on OS/400 to resolve a const/nonconst parameter problem. - [Patrick Monnerat brought this change] Add interface for OS/400 crypto library QC3 - [Patrick Monnerat brought this change] misc: include stdarg.h for debug code - [Patrick Monnerat brought this change] Document crypto library interface - [Patrick Monnerat brought this change] Feature an optional crypto-specific macro to rsa sign a data fragment vector OS/400 crypto library is unable to sign a precomputed SHA1 hash: however it does support a procedure that hashes data fragments and rsa signs. If defined, the new macro _libssh2_rsa_sha1_signv() implements this function and disables use of _libssh2_rsa_sha1_sign(). The function described above requires that the struct iovec unused slacks are cleared: for this reason, macro libssh2_prepare_iovec() has been introduced. It should be defined as empty for crypto backends that are not sensitive to struct iovec unused slack values. - [Patrick Monnerat brought this change] Fold long lines in include files - [Viktor Szakats brought this change] kex.c: fix indentation Closes #71 - [Viktor Szakats brought this change] add OpenSSL-1.1.0-pre2 compatibility Closes #70 - [Viktor Szakats brought this change] add OpenSSL 1.1.0-pre1 compatibility * close https://github.com/libssh2/libssh2/issues/69 * sync a declaration with the rest of similar ones * handle EVP_MD_CTX_new() returning NULL with OpenSSL 1.1.0 * fix potential memory leak with OpenSSL 1.1.0 in _libssh2_*_init() functions, when EVP_MD_CTX_new() succeeds, but EVP_DigestInit() fails. Marc Hoersken (22 Dec 2015) - wincng.c: fixed _libssh2_wincng_hash_final return value _libssh2_wincng_hash_final was returning the internal BCRYPT status code instead of a valid libssh2 return value (0 or -1). This also means that _libssh2_wincng_hash never returned 0. - wincng.c: fixed possible memory leak in _libssh2_wincng_hash If _libssh2_wincng_hash_update failed _libssh2_wincng_hash_final would never have been called before. Reported by Zenju. Kamil Dudka (15 Dec 2015) - [Paul Howarth brought this change] libssh2.pc.in: fix the output of pkg-config --libs ... such that it does not include LDFLAGS used to build libssh2 itself. There was a similar fix in the curl project long time ago: https://github.com/bagder/curl/commit/curl-7_19_7-56-g4c8adc8 Bug: https://bugzilla.redhat.com/1279966 Signed-off-by: Kamil Dudka Marc Hoersken (6 Dec 2015) - hostkey.c: align code path of ssh_rsa_init to ssh_dss_init - hostkey.c: fix invalid memory access if libssh2_dsa_new fails Reported by dimmaq, fixes #66 Daniel Stenberg (3 Nov 2015) - [Will Cosgrove brought this change] gcrypt: define libssh2_sha256_ctx Looks like it didn't make it into the latest commit for whatever reason. Closes #58 - [Salvador Fandino brought this change] libssh2_session_set_last_error: Add function Net::SSH2, the Perl wrapping module for libssh2 implements several features* on top of libssh2 that can fail and so need some mechanism to report the error condition to the user. Until now, besides the error state maintained internally by libssh2, another error state was maintained at the Perl level for every session object and then additional logic was used to merge both error states. That is a maintenance nighmare, and actually there is no way to do it correctly and consistently. In order to allow the high level language to add new features to the library but still rely in its error reporting features the new function libssh2_session_set_last_error (that just exposses _libssh2_error_flags) is introduced. *) For instance, connecting to a remote SSH service giving the hostname and port. Signed-off-by: Salvador Fandino Signed-off-by: Salvador Fandiño - [Salvador Fandino brought this change] _libssh2_error: Support allocating the error message Before this patch "_libssh2_error" required the error message to be a static string. This patch adds a new function "_libssh2_error_flags" accepting an additional "flags" argument and specifically the flag "LIBSSH2_ERR_FLAG_DUP" indicating that the passed string must be duplicated into the heap. Then, the method "_libssh2_error" has been rewritten to use that new function under the hood. Signed-off-by: Salvador Fandino Signed-off-by: Salvador Fandiño - [Will Cosgrove brought this change] added engine.h include to fix warning - [sune brought this change] kex.c: removed dupe entry from libssh2_kex_methods[] Closes #51 - [Salvador Fandiño brought this change] userauth: Fix off by one error when reading public key file After reading the public key from file the size was incorrectly decremented by one. This was usually a harmless error as the last character on the public key file is an unimportant EOL. But if due to some error the public key file is empty, the public key size becomes (uint)(0 - 1), resulting in an unrecoverable out of memory error later. Signed-off-by: Salvador Fandi??o - [Salvador Fandino brought this change] channel: Detect bad usage of libssh2_channel_process_startup A common novice programmer error (at least among those using the wrapping Perl module Net::SSH2), is to try to reuse channels. This patchs detects that incorrect usage and fails with a LIBSSH2_ERROR_BAD_USE error instead of hanging. Signed-off-by: Salvador Fandino - [Will Cosgrove brought this change] kex: Added diffie-hellman-group-exchange-sha256 support ... and fixed HMAC_Init depricated usage Closes #48 Alexander Lamaison (21 Sep 2015) - Prefixed new #defines to prevent collisions. Other libraries might have their own USE_WIN32_*FILES. - [keith-daigle brought this change] Update examples/scp.c to fix bug where large files on win32 would cause got to wrap and go negative - [David Byron brought this change] add libssh2_scp_recv2 to support large (> 2GB) files on windows Daniel Stenberg (17 Sep 2015) - [sune brought this change] WinCNG: support for SHA256/512 HMAC Closes #47 - [brian m. carlson brought this change] Add support for HMAC-SHA-256 and HMAC-SHA-512. Implement support for these algorithms and wire them up to the libgcrypt and OpenSSL backends. Increase the maximum MAC buffer size to 64 bytes to prevent buffer overflows. Prefer HMAC-SHA-256 over HMAC-SHA-512, and that over HMAC-SHA-1, as OpenSSH does. Closes #40 - [Zenju brought this change] kex: free server host key before allocating it (again) Fixes a memory leak when Synology server requests key exchange Closes #43 - [Viktor Szakats brought this change] GNUmakefile: up OpenSSL version closes #23 - [Viktor Szakats brought this change] GNUmakefile: add -m64 CFLAGS when targeting mingw64, add -m32/-m64 to LDFLAGS libssh2 equivalent of curl patch https://github.com/bagder/curl/commit/d21b66835f2af781a3c2a685abc92ef9f0cd86be This allows to build for the non-default target when using a multi-target mingw distro. Also bump default OpenSSL dependency path to 1.0.2c. - [Viktor Szakats brought this change] GNUmakefile: add support for LIBSSH2_LDFLAG_EXTRAS It is similar to existing LIBSSH2_CFLAG_EXTRAS, but for extra linker options. Also delete some line/file ending whitespace. closes #27 - [nasacj brought this change] hostkey.c: Fix compiling error when OPENSSL_NO_MD5 is defined Closes #32 - [Mizunashi Mana brought this change] openssl.h: adjust the rsa/dsa includes ... to work when built without DSA support. Closes #36 Alexander Lamaison (26 Jul 2015) - Let CMake build work as a subproject. Patch contributed by JasonHaslam. - Fix builds with Visual Studio 2015. VS2015 moved stdio functions to the header files as inline function. That means check_function_exists can't detect them because it doesn't use header files - just does a link check. Instead we need to use check_symbol_exists with the correct headers. Kamil Dudka (2 Jul 2015) - cmake: include CMake files in the release tarballs Despite we announced the CMake support in libssh2-1.6.0 release notes, the files required by the CMake build system were not included in the release tarballs. Hence, the only way to use CMake for build was the upstream git repository. This commit makes CMake actually supported in the release tarballs. - tests/mansyntax.sh: fix 'make distcheck' with recent autotools Do not create symbolic links off the build directory. Recent autotools verify that out-of-source build works even if the source directory tree is not writable. - openssl: fix memleak in _libssh2_dsa_sha1_verify() Daniel Stenberg (12 Jun 2015) - openssl: make libssh2_sha1 return error code - use the internal prefix _libssh2_ for non-exported functions - removed libssh2_md5() since it wasn't used Reported-by: Kamil Dudka - [LarsNordin-LNdata brought this change] SFTP: Increase speed and datasize in SFTP read The function sftp_read never return more then 2000 bytes (as it should when I asked Daniel). I increased the MAX_SFTP_READ_SIZE to 30000 but didn't get the same speed as a sftp read in SecureSSH. I analyzed the code and found that a return always was dona when a chunk has been read. I changed it to a sliding buffer and worked on all available chunks. I got an increase in speed and non of the test I have done has failed (both local net and over Internet). Please review and test. I think 30000 is still not the optimal MAX_SFTP_READ_SIZE, my next goal is to make an API to enable changing this value (The SecureSSH sftp_read has more complete filled packages when comparing the network traffic) - bump: start working on 1.6.1 Version 1.6.0 (5 Jun 2015) Daniel Stenberg (5 Jun 2015) - RELEASE-NOTES: synced with 858930cae5c6a Marc Hoersken (19 May 2015) - wincng.c: fixed indentation - [sbredahl brought this change] wincng.c: fixed memleak in (block) cipher destructor Alexander Lamaison (6 May 2015) - [Jakob Egger brought this change] libssh2_channel_open: more detailed error message The error message returned by libssh2_channel_open in case of a server side channel open failure is now more detailed and includes the four standard error conditions in RFC 4254. - [Hannes Domani brought this change] kex: fix libgcrypt memory leaks of bignum Fixes #168. Marc Hoersken (3 Apr 2015) - configure.ac: check for SecureZeroMemory for clear memory feature - Revert "wincng.c: fix clear memory feature compilation with mingw" This reverts commit 2d2744efdd0497b72b3e1ff6e732aa4c0037fc43. Autobuilds show that this did not solve the issue. And it seems like RtlFillMemory is defined to memset, which would be optimized out by some compilers. - wincng.c: fix clear memory feature compilation with mingw Alexander Lamaison (1 Apr 2015) - [LarsNordin-LNdata brought this change] Enable use of OpenSSL that doesn't have DSA. Added #if LIBSSH2_DSA for all DSA functions. - [LarsNordin-LNdata brought this change] Use correct no-blowfish #define with OpenSSL. The OpenSSL define is OPENSSL_NO_BF, not OPENSSL_NO_BLOWFISH. Marc Hoersken (25 Mar 2015) - configure: error if explicitly enabled clear-memory is not supported This takes 22bd8d81d8fab956085e2079bf8c29872455ce59 and b8289b625e291bbb785ed4add31f4759241067f3 into account, but still makes it enabled by default if it is supported and error out in case it is unsupported and was requested. Daniel Stenberg (25 Mar 2015) - configure: make clear-memory default but only WARN if backend unsupported ... instead of previous ERROR. Marc Hoersken (24 Mar 2015) - wincng.h: fix warning about computed return value not being used - nonblocking examples: fix warning about unused tvdiff on Mac OS X Daniel Stenberg (24 Mar 2015) - openssl: fix compiler warnings - cofigure: fix --disable-clear-memory check Marc Hoersken (23 Mar 2015) - scp.c: improved command length calculation Reduced number of calls to strlen, because shell_quotearg already returns the length of the resulting string (e.q. quoted path) which we can add to the existing and known cmd_len. Removed obsolete call to memset again, because we can put a final NULL-byte at the end of the string using the calculated length. - scp.c: improved and streamlined formatting - scp.c: fix that scp_recv may transmit not initialised memory - scp.c: fix that scp_send may transmit not initialised memory Fixes ticket 244. Thanks Torsten. - kex: do not ignore failure of libssh2_sha1_init() Based upon 43b730ce56f010e9d33573fcb020df49798c1ed8. Fixes ticket 290. Thanks for the suggestion, mstrsn. - wincng.h: fix return code of libssh2_md5_init() - openssl.c: fix possible segfault in case EVP_DigestInit fails - wincng.c: fix possible use of uninitialized variables - wincng.c: fix unused argument warning if clear memory is not enabled - wincng: Added explicit clear memory feature to WinCNG backend This re-introduces the original feature proposed during the development of the WinCNG crypto backend. It still needs to be added to libssh2 itself and probably other backends. Memory is cleared using the function SecureZeroMemory which is available on Windows systems, just like the WinCNG backend. - wincng.c: fixed mixed line-endings - wincng.c: fixed use of invalid parameter types in a8d14c5dcf - wincng.c: only try to load keys corresponding to the algorithm - wincng.c: moved PEM headers into definitions - wincng.h: fixed invalid parameter name - wincng: fixed mismatch with declarations in crypto.h - userauth.c: fixed warning C6001: using uninitialized sig and sig_len - pem.c: fixed warning C6269: possible incorrect order of operations - wincng: add support for authentication keys to be passed in memory Based upon 18cfec8336e and daa2dfa2db. - pem.c: add _libssh2_pem_parse_memory to parse PEM from memory Requirement to implement 18cfec8336e for Libgcrypt and WinCNG. - pem.c: fix copy and paste mistake from 55d030089b8 - userauth.c: fix another possible dereference of a null pointer - userauth.c: fix possible dereference of a null pointer - pem.c: reduce number of calls to strlen in readline Alexander Lamaison (17 Mar 2015) - [Will Cosgrove brought this change] Initialise HMAC_CTX in more places. Missed a couple more places we init ctx to avoid openssl threading crash. - Build build breakage in WinCNG backend caused when adding libssh2_userauth_publickey_frommemory. The new feature isn't implemented for the WinCNG backend currently, but the WinCNG backend didn't contain any implementation of the required backend functions - even ones that returns an error. That caused link errors. This change fixes the problem by providing an implementation of the backend functions that returns an error. - Fix breakage in WinCNG backend caused by introducing libssh2_hmac_ctx_init. The macro was defined to nothing for the libgcrypt backend, but not for WinCNG. This brings the latter into line with the former. Daniel Stenberg (15 Mar 2015) - userauth_publickey_frommemory.3: add AVAILABILITY ... it will be added in 1.6.0 - libssh2: next version will be called 1.6.0 ... since we just added a new function. - docs: add libssh2_userauth_publickey_frommemory.3 to dist The function and man page were added in commit 18cfec8336e - [Jakob Egger brought this change] direct_tcpip: Fixed channel write There were 3 bugs in this loop: 1) Started from beginning after partial writes 2) Aborted when 0 bytes were sent 3) Ignored LIBSSH2_ERROR_EAGAIN See also: https://trac.libssh2.org/ticket/281 https://trac.libssh2.org/ticket/293 Alexander Lamaison (15 Mar 2015) - [Will Cosgrove brought this change] Must init HMAC_CTX before using it. Must init ctx before using it or openssl will reuse the hmac which is not thread safe and causes a crash. Added libssh2_hmac_ctx_init macro. - Add continuous integration configurations. Linux-based CI is done by Travis CI. Windows-based CI is done by Appveyor. - [David Calavera brought this change] Allow authentication keys to be passed in memory. All credits go to Joe Turpin, I'm just reaplying and cleaning his patch: http://www.libssh2.org/mail/libssh2-devel-archive-2012-01/0015.shtml * Use an unimplemented error for extracting keys from memory with libgcrypt. Daniel Stenberg (14 Mar 2015) - docs: include the renamed INSTALL* files in dist Alexander Lamaison (13 Mar 2015) - Prevent collisions between CMake and Autotools in examples/ and tests/. - Avoid clash between CMake build and Autotools. Autotools expects a configuration template file at src/libssh2_config.h.in, which buildconf generates. But the CMake build system has its CMake-specific version of the file at this path. This means that, if you don't run buildconf, the Autotools build will fail because it configured the wrong header template. See https://github.com/libssh2/libssh2/pull/8. - Merge pull request #8 from alamaison/cmake CMake build system. - CMake build system. Tested: - Windows: - Visual C++ 2005/2008/2010/2012/2013/MinGW-w64 - static/shared - 32/64-bit - OpenSSL/WinCNG - Without zlib - Linux: - GCC 4.6.3/Clang 3.4 - static/shared - 32/64-bit - OpenSSL/Libgcrypt - With/Without zlib - MacOS X - AppleClang 6.0.0 - static - 64-bit - OpenSSL - Without zlib Conflicts: README - Man man syntax tests fail gracefully if man version is not suitable. - Return valid code from test fixture on failure. The sshd test fixture was returning -1 if an error occurred, but negative error codes aren't technically valid (google it). Bash on Windows converted them to 0 which made setup failure look as though all tests were passing. - Let mansyntax.sh work regardless of where it is called from. Daniel Stenberg (12 Mar 2015) - [Viktor Szakáts brought this change] mingw build: allow to pass custom CFLAGS Allow to pass custom `CFLAGS` options via environment variable `LIBSSH2_CFLAG_EXTRAS`. Default and automatically added options of `GNUmakefile` have preference over custom ones. This addition is useful for passing f.e. custom CPU tuning or LTO optimization (`-flto -ffat-lto-objects`) options. The only current way to do this is to edit `GNUmakefile`. This patch makes it unnecessary. This is a mirror of similar libcurl patch: https://github.com/bagder/curl/pull/136 - [Will Cosgrove brought this change] userauth: Fixed prompt text no longer being copied to the prompts struct Regression from 031566f9c - README: update the git repo locations - wait_socket: wrong use of difftime() With reversed arguments it would always return a negative value... Bug: https://github.com/bagder/libssh2/issues/1 - bump: start working toward 1.5.1 now Version 1.5.0 (11 Mar 2015) Daniel Stenberg (11 Mar 2015) - RELEASE-NOTES: 1.5.0 release - [Mariusz Ziulek brought this change] kex: bail out on rubbish in the incoming packet CVE-2015-1782 Bug: http://www.libssh2.org/adv_20150311.html - docs: move INSTALL, AUTHORS, HACKING and TODO to docs/ And with this, cleanup README to be shorter and mention the new source code home. - .gitignore: don't ignore INSTALL Dan Fandrich (4 Mar 2015) - examples/x11.c: include sys/select.h for improved portability Daniel Stenberg (4 Mar 2015) - RELEASE-NOTES: synced with a8473c819bc068 In preparation for the upcoming 1.5.0 release. Guenter Knauf (8 Jan 2015) - NetWare build: added some missing exports. Marc Hoersken (29 Dec 2014) - knownhost.c: fix use of uninitialized argument variable wrote Detected by clang scan in line 1195, column 18. - examples/x11.c: fix result of operation is garbage or undefined Fix use of uninitialized structure w_size_bck. Detected by clang scan in line 386, column 28. - examples/x11.c: remove dead assigments of some return values Detected by clang scan in line 212, column 9. Detected by clang scan in line 222, column 13. Detected by clang scan in line 410, column 13. - examples/x11.c: fix possible memory leak if read fails Detected by clang scan in line 224, column 21. - examples/x11.c: fix invalid removal of first list element Fix use of memory after it was being freed. Detected by clang scan in line 56, column 12. - userauth.c: make sure that sp_len is positive and avoid overflows ... if the pointer subtraction of sp1 - pubkey - 1 resulted in a negative or larger value than pubkey_len, memchr would fail. Reported by Coverity CID 89846. - channel.c: remove logically dead code, host cannot be NULL here ... host cannot be NULL in line 525, because it is always valid (e.g. at least set to "0.0.0.0") after lines 430 and 431. Reported by Coverity CID 89807. - session.c: check return value of session_nonblock during startup Reported by Coverity CID 89803. - session.c: check return value of session_nonblock in debug mode Reported by Coverity CID 89805. - pem.c: fix mixed line-endings introduced with 8670f5da24 - pem.c: make sure there's a trailing zero and b64data is not NULL ... if there is no base64 data between PEM header and footer. Reported by Coverity CID 89823. - kex.c: make sure mlist is not set to NULL ... if the currently unsupported LANG methods are called. Reported by Coverity CID 89834. - packet.c: i < 256 was always true and i would overflow to 0 Visualize that the 0-termination is intentional, because the array is later passed to strlen within _libssh2_packet_askv. - silence multiple data conversion warnings Daniel Stenberg (23 Dec 2014) - agent_connect_unix: make sure there's a trailing zero ... if the path name was too long. Reported by Coverity CID 89801. Marc Hoersken (22 Dec 2014) - examples on Windows: use native SOCKET-type instead of int And check return values accordingly. - userauth.c: improve readability and clarity of for-loops Daniel Stenberg (22 Dec 2014) - calloc: introduce LIBSSH2_CALLOC() A simple function using LIBSSH2_ALLOC + memset, since this pattern was used in multiple places and this simplies code in general. Marc Hoersken (15 Dec 2014) - libssh2_priv.h: Ignore session, context and format parameters - x11 example: check return value of socket function - examples: fixed mixed line-endings introduced with aedfba25b8 - wincng.c: explicitly ignore BCrypt*AlgorithmProvider return codes Fixes VS2012 code analysis warning C6031: return value ignored: could return unexpected value - wincng.c: fix possible invalid memory write access Fixes VS2012 code analysis warning C6386: buffer overrun: accessing 'pbOutput', the writable size is 'cbOutput' bytes, but '3' bytes may be written: libssh2 wincng.c 610 - tests on Windows: check for WSAStartup return code Fixes VS2012 code analysis warning C6031: return value ignored: could return unexpected value - wincng.c: fix possible NULL pointer de-reference of bignum Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'bignum'. libssh2 wincng.c 1567 - wincng.c: fix possible use of uninitialized memory Fixes VS2012 code analysis warning C6001: using uninitialized memory 'cbDecoded'. libssh2 wincng.c 553 - packet.c: fix possible NULL pointer de-reference within listen_state Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'listen_state->channel'. libssh2 packet.c 221 - kex.c: fix possible NULL pointer de-reference with session->kex Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'session->kex'. libssh2 kex.c 1761 - agent.c: check return code of MapViewOfFile Fixes VS2012 code analysis warning C6387: 'p+4' may be '0': this does not adhere to the specification for the function 'memcpy': libssh2 agent.c 330 Fixes VS2012 code analysis warning C6387: 'p' may be '0': this does not adhere to the specification for the function 'UnmapViewOfFile': libssh2 agent.c 333 - examples on Windows: check for socket return code Fixes VS2012 code analysis warning C28193: The variable holds a value that must be examined - examples on Windows: check for WSAStartup return code Fixes VS2012 code analysis warning C6031: return value ignored: could return unexpected value Guenter Knauf (11 Dec 2014) - wincng.c: silent some more gcc compiler warnings. - wincng.c: silent gcc compiler warnings. - Watcom build: added support for WinCNG build. - build: updated dependencies in makefiles. Daniel Stenberg (4 Dec 2014) - configure: change LIBS not LDFLAGS when checking for libs Closes #289 Patch-by: maurerpe Guenter Knauf (3 Dec 2014) - MinGW build: some more GNUMakefile tweaks. test/GNUmakefile: added architecture autodetection; added switches to CFLAGS and RCFLAGS to make sure that the right architecture is used. Added support to build with WinCNG. - sftpdir.c: added authentication method detection. Stuff copied over from ssh2.c to make testing a bit easier. - NMake build: fixed LIBS settings. - NMake build: added support for WinCNG build. - MinGW build: some GNUMakefile tweaks. Added architecture autodetection; added switches to CFLAGS and RCFLAGS to make sure that the right architecture is used. Added support to build with WinCNG. - MinGW build: Fixed redefine warnings. - Updated copyright year. Daniel Stenberg (31 Aug 2014) - COPYING: bump the copyright year Dan Fandrich (28 Jul 2014) - docs: fixed a bunch of typos - docs: added missing libssh2_session_handshake.3 file Marc Hoersken (19 May 2014) - wincng.c: specify the required libraries for dependencies using MSVC Initially reported by Bob Kast as "for MS VS builds, specify the libraries that are required so they don't need to go into all project files that may use this library". Thanks a lot. - [Bob Kast brought this change] windows build: do not export externals from static library If you are building a DLL, then you need to explicitly export each entry point. When building a static library, you should not. libssh2 was exporting the entry points whether it was building a DLL or a static library. To elaborate further, if libssh2 was used as a static library, which was being linked into a DLL, the libssh2 API would be exported from that separate DLL. Daniel Stenberg (19 May 2014) - [Mikhail Gusarov brought this change] Fix typos in manpages Marc Hoersken (18 May 2014) - wincng.c: Fixed memory leak in case of an error during ASN.1 decoding - configure: Display individual crypto backends on separate lines This avoids line-wrapping in between parameters and makes the error message look like the following: configure: error: No crypto library found! Try --with-libssl-prefix=PATH or --with-libgcrypt-prefix=PATH or --with-wincng on Windows - [Bob Kast brought this change] libssh2_priv.h: a 1 bit bit-field should be unsigned some compilers may not like this - knownhost.c: Fixed warning that pointer targets differ in signedness - wincng.c: Fixed warning about pointer targets differing in signedness - tcpip-forward.c: Fixed warning that pointer targets differ in signedness libssh2_channel_forward_listen_ex uses ints instead of unsigned ints. - misc.c: Fixed warning about mixed declarations and code - libgcrypt.h: Fixed warning about pointer targets differing in signedness - wincng.h: Fixed warning about pointer targets differing in signedness - misc.c: Fixed warning about unused parameter abstract - tcpip-forward.c: Removed unused variables shost, sport and sockopt - wincng.h: Added forward declarations for all WinCNG functions Initially reported by Bob Kast as "Wincng - define function prototypes for wincng routines". Thanks a lot. Also replaced structure definitions with type definitions. - [Bob Kast brought this change] libssh2.h: on Windows, a socket is of type SOCKET, not int - win32: Added WinCNG targets to generated Visual Studio project Inspired by Bob Kast's reports, this commit enables the compilation of libssh2 with WinCNG using the generated Visual Studio project files. This commit adds WinCNG support to parts of the existing Win32 build infrastructure, until new build systems, like pre-defined VS project files or CMake files may be added. This commit and b20bfeb3e519119a48509a1099c06d65aa7da1d7 raise one question: How to handle build systems, like VS project files, that need to include all source files regardless of the desired target, including all supported crypto backends? For now the mentioned commit added a check for LIBSSH2_OPENSSL to openssl.c and with this commit the supported crypto backends are hardcoded within Makefile.am. - libssh2_priv msvc: Removed redundant definition of inline keyword Initially reported by Bob Kast as "Remove redundant 'inline' define". Thanks a lot. - wincng: Made data parameter to hash update function constant Initially reported by Bob Kast as "formal parameter must be const since it is used in contexts where the actual parameter may be const". Thanks a lot. - wincng: fix cross-compilation against the w64 mingw-runtime package - openssl: Check for LIBSSH2_OPENSSL in order to compile with openssl - wincng: Fixed use of possible uninitialized variable pPaddingInfo Reported by Bob Kast, thanks a lot. - wincng: Added cast for double to unsigned long conversion - wincng: Cleaned up includes and check NTSTATUS using macro Removed header file combination that is not supported on a real Windows platform and can only be compiled using MinGW. Replaced custom NTSTATUS return code checks with BCRYPT_SUCCESS macro. Daniel Stenberg (16 Mar 2014) - userauth_hostbased_fromfile: zero assign to avoid uninitialized use Detected by clang-analyze - channel_receive_window_adjust: store windows size always Avoid it sometimes returning without storing it, leaving calling functions with unknown content! Detected by clang-analyzer - publickey_packet_receive: avoid junk in returned pointers clang-analyzer found this risk it would return a non-initialized pointer in a success case Peter Stuge (16 Mar 2014) - [Marc Hoersken brought this change] Added Windows Cryptography API: Next Generation based backend - [Marc Hoersken brought this change] knownhost.c: fixed that 'key_type_len' may be used uninitialized ../src/knownhost.c: In function 'libssh2_knownhost_readline': ../src/knownhost.c:651:16: warning: 'key_type_len' may be used uninitialized in this function [-Wmaybe-uninitialized] rc = knownhost_add(hosts, hostbuf, NULL, ^ ../src/knownhost.c:745:12: note: 'key_type_len' was declared here size_t key_type_len; ^ - [Marc Hoersken brought this change] pem.c: always compile pem.c independently of crypto backend - Fix non-autotools builds: Always define the LIBSSH2_OPENSSL CPP macro Commit d512b25f69a1b6778881f6b4b5ff9cfc6023be42 introduced a crypto library abstraction in the autotools build system, to allow us to more easily support new crypto libraries. In that process it was found that all other build system which we support are hard-coded to build with OpenSSL. Commit f5c1a0d98bd51aeb24aca3d49c7c81dcf8bd858d fixes automake introduced into non-autotools build systems but still overlooked the CPP macro saying that we are using OpenSSL. Thanks to Marc Hörsken for identifying this issue and proposing a fix for win32/{GNUmakefile,config.mk}. This commit uses a slightly different approach but the end result is the same. Dan Fandrich (15 Mar 2014) - channel_close: Close the channel even in the case of errors - sftp_close_handle: ensure the handle is always closed Errors are reported on return, but otherwise the close path is completed as much as possible and the handle is freed on exit. Alexander Lamaison (6 Mar 2014) - knownhost: Restore behaviour of `libssh2_knownhost_writeline` with short buffer. Commit 85c6627c changed the behaviour of `libssh2_knownhost_writeline` so that it stopped returning the number of bytes needed when the given buffer was too small. Also, the function changed such that is might write to part of the buffer before realising it is too small. This commit restores the original behaviour, whilst keeping the unknown-key-type functionality that 85c6627c. Instead of writing to the buffer piecemeal, the length of the various parts is calculated up front and the buffer written only if there is enough space. The calculated necessary size is output in `outlen` regardless of whether the buffer was written to. The main use-case for the original behaviour that this commit restores is to allow passing in a NULL buffer to get the actual buffer size needed, before calling the function again with the buffer allocated to the exact size required. - knownhost: Fix DSS keys being detected as unknown. I missing `else` meant ssh-dss format keys were being re-detected as unknown format. Dan Fandrich (6 Mar 2014) - knownhosts: Abort if the hosts buffer is too small This could otherwise cause a match on the wrong host - agent_list_identities: Fixed memory leak on OOM - Fixed a few typos - userauth: Fixed an attempt to free from stack on error - Fixed a few memory leaks in error paths - Fixed two potential use-after-frees of the payload buffer The first might occur if _libssh2_packet_add returns an error, as fullpacket_state wasn't reset to idle so if it were possible for fullpacket to be called again, it would return to the same state handler and re-use the freed p->packet buffer. The second could occur if decrypt returned an error, as it freed the packet buffer but did not clear total_num, meaning that freed buffer could be written into again later. Alexander Lamaison (28 Nov 2013) - Fix missing `_libssh2_error` in `_libssh2_channel_write`. In one case, the error code from `_libssh2_transport_read` was being returned from `_libssh2_channel_write` without setting it as the last error by calling `_libssh2_error`. This commit fixes that. Found when using a session whose socket had been inadvertently destroyed. The calling code got confused because via `libssh2_session_last_error` it appeared no error had occurred, despite one being returned from the previous function. Kamil Dudka (21 Nov 2013) - [Mark McPherson brought this change] openssl: initialise the digest context before calling EVP_DigestInit() When using the OpenSSL libraries in FIPS mode, the function call EVP_DigestInit() is actually #defined to FIPS_digestinit(). Unfortunately wheres EVP_DigestInit() initialises the context and then calls EVP_DigestInit_ex(), this function assumes that the context has been pre-initialised and crashes when it isn't. Bug: https://trac.libssh2.org/ticket/279 Fixes #279 - [Marc Hörsken brought this change] .gitignore: Ignore files like src/libssh2_config.h.in~ Peter Stuge (13 Nov 2013) - Move automake conditionals added by commit d512b25f out of Makefile.inc Commit d512b25f69a1b6778881f6b4b5ff9cfc6023be42 added automake conditionals to Makefile.inc but since Makefile.inc is included from Makefile for all other build systems that does not work. This commit instead adds Makefile.OpenSSL.inc and Makefile.libgcrypt.inc and moves the automake conditional to its proper place, src/Makefile.am. The automake conditional includes the correct Makefile.$name.inc per the crypto library selection/detection done by configure. All non-autotools build system files in libssh2 are hardcoded to use OpenSSL and do not get a conditional but at least there is some reuse because they can all include the new Makefile.OpenSSL.inc. Daniel Stenberg (27 Oct 2013) - [Salvador Fandino brought this change] Set default window size to 2MB The default channel window size used until now was 256KB. This value is too small and results on a bottleneck on real-life networks where round-trip delays can easily reach 300ms. The issue was not visible because the configured channel window size was being ignored and a hard-coded value of ~22MB being used instead, but that was fixed on a previous commit. This patch just changes the default window size (LIBSSH2_CHANNEL_WINDOW_DEFAULT) to 2MB. It is the same value used by OpenSSH and in our opinion represents a good compromise between memory used and transfer speed. Performance tests were run to determine the optimum value. The details and related discussion are available from the following thread on the libssh2 mailing-list: http://www.libssh2.org/mail/libssh2-devel-archive-2013-10/0018.shtml http://article.gmane.org/gmane.network.ssh.libssh2.devel/6543 An excerpt follows: "I have been running some transfer test and measuring their speed. My setup was composed of a quad-core Linux machine running Ubuntu 13.10 x86_64 with a LXC container inside. The data transfers were performed from the container to the host (never crossing through a physical network device). Network delays were simulated using the tc tool. And ping was used to verify that they worked as intended during the tests. The operation performed was the equivalent to the following ssh command: $ ssh container "dd bs=16K count=8K if=/dev/zero" >/dev/null Though, establishment and closing of the SSH connection was excluded from the timings. I run the tests several times transferring files of sizes up to 128MB and the results were consistent between runs. The results corresponding to the 128MB transfer are available here: https://docs.google.com/spreadsheet/ccc?key=0Ao1yRmX6PQQzdG5wSFlrZl9HRWNET3ZyN0hnaGo5ZFE&usp=sharing It clearly shows that 256KB is too small as the default window size. Moving to a 512MB generates a great improvement and after the 1MB mark the returns rapidly diminish. Other factors (TCP window size, probably) become more limiting than the channel window size For comparison I also performed the same transfers using OpenSSH. Its speed is usually on par with that of libssh2 using a window size of 1MB (even if it uses a 2MB window, maybe it is less aggressive sending the window adjust msgs)." Signed-off-by: Salvador Fandino - [Salvador brought this change] _libssh2_channel_read: Honour window_size_initial _libssh2_channel_read was using an arbitrary hard-coded limit to trigger the window adjusting code. The adjustment used was also hard-coded and arbitrary, 15MB actually, which would limit the usability of libssh2 on systems with little RAM. This patch, uses the window_size parameter passed to libssh2_channel_open_ex (stored as remote.window_size_initial) plus the buflen as the base for the trigger and the adjustment calculation. The memory usage when using the default window size is reduced from 22MB to 256KB per channel (actually, if compression is used, these numbers should be incremented by ~50% to account for the errors between the decompressed packet sizes and the predicted sizes). My tests indicate that this change does not impact the performance of transfers across localhost or a LAN, being it on par with that of OpenSSH. On the other hand, it will probably slow down transfers on networks with high bandwidth*delay when the default window size (LIBSSH2_CHANNEL_WINDOW_DEFAULT=256KB) is used. Signed-off-by: Salvador Fandino - [Salvador Fandino brought this change] knownhosts: handle unknown key types Store but don't use keys of unsupported types on the known_hosts file. Currently, when libssh2 parses a known_host file containing keys of some type it doesn't natively support, it stops reading the file and returns an error. That means, that the known_host file can not be safely shared with other software supporting other key types (i.e. OpenSSH). This patch adds support for handling keys of unknown type. It can read and write them, even if they are never going to be matched. At the source level the patch does the following things: - add a new unknown key type LIBSSH2_KNOWNHOST_KEY_UNKNOWN - add a new slot (key_type_name) on the known_host struct that is used to store the key type in ascii form when it is not supported - parse correctly known_hosts entries with unknown key types and populate the key_type_name slot - print correctly known_hosts entries of unknown type - when checking a host key ignore keys that do not match the key Fixes #276 - windows build: fix build errors Fixes various link errors with VS2010 Reported-by: "kdekker" Fixes #272 - man page: add missing function argument for libssh2_userauth_publickey_fromfile_ex() Reported-by: "pastey" Fixes #262 - [Salvador brought this change] Fix zlib deflate usage Deflate may return Z_OK even when not all data has been compressed if the output buffer becomes full. In practice this is very unlikely to happen because the output buffer size is always some KBs larger than the size of the data passed for compression from the upper layers and I think that zlib never expands the data so much, even on the worst cases. Anyway, this patch plays on the safe side checking that the output buffer is not exhausted. Signed-off-by: Salvador