GFAL2 RELEASE-NOTES =================== ## Tue Feb 07 2023 Joao Lopes <batistal at cern.ch> - 2.21.3 ###New Feature - [DMC-1290] - Gfal2 Mock Plugin support for Archive operation - [DMC-1357] - Report source and destination hosts involved in HTTP Third Party Copy - [DMC-1369] - Make Gfal2 HTTP "HEADERS" option configurable per Storage Endpoint ###Task - [DMC-1359] - Data Management Clients for Fedora 37 ###Improvement - [DMC-1362] - Gfal2 HTTP logging should respect the "LOG_LEVEL" configuration - [DMC-1365] - Protect gfal_xrootd_bring_online_list() from reading un-allocated memory - [DMC-1366] - Two GFAL2 XRootD bring on-line error messages have the same message ## Thu Dec 01 2022 Mihai Patrascoiu <mipatras at cern.ch> - 2.21.2 ###New Feature - [DMC-1353] - Always return Adler32 checksum as 8-byte string - [DMC-1354] - Make Gfal2 HTTP "ENABLE_REMOTE_COPY" option configurable per Storage Endpoint - [gfal2/pull/53] - Allow the use of roots:// and xroots:// schema for XRootd + TLS ###Task - [DMC-1351] - Pass bringonline token to evict call inside HTTP copy ###Improvement - [DMC-1350] - Reduce metalink requests in Gfal2 HTTP plugin - [DMC-1355] - Disable metalink support by default for Gfal2 transfers ## Thu Sep 29 2022 Mihai Patrascoiu <mipatras at cern.ch> - 2.21.1 ###Bug - [DMC-1334] - Gfal2 HTTP bring online polling missing files ###New Feature - [DMC-1347] - Allow GFAL2 ENABLE_FALLBACK_TPC_COPY to be overridden per SE - [DMC-1349] - Report IPv4 transfer event ###Task - [DMC-1327] - DMC packages for Centos Stream 9 - [DMC-1329] - Create "taperestapi.uri" extended attribute in Gfal2 HTTP plugin - [DMC-1331] - Create "taperestapi.sitename" extended attribute in Gfal2 HTTP plugin - [DMC-1336] - Data Management Clients for Fedora 36 ###Improvement - [DMC-1339] - HTTP streamed copy should apply the transfer timeout to both source and destination operations - [DMC-1340] - Better reporting of errors during HTTP streamed transfers ## Fri Jul 08 2022 Joao Lopes <batistal at cern.ch> - 2.21.0 ###Epic - [DMC-1301] - Gfal2 Support for HTTP Tape Operations ###New Feature - [DMC-1302] - Implement HTTP bringonline operation - [DMC-1303] - Implement HTTP bringonline polling operation - [DMC-1304] - Implement HTTP archive polling operation - [DMC-1305] - Implement HTTP release-file operation - [DMC-1306] - Implement HTTP eviction on successful copy operation - [DMC-1307] - Implement HTTP Tape Endpoint discovery - [DMC-1311] - Report IP stack used during HTTP-TPC in Gfal2 - [DMC-1314] - Implement HTTP bringonline cancel operation - [DMC-1316] - Allow configuration of HTTP "ENABLE_STREAM_COPY" parameter by Storage Endpoint - [DMC-1321] - HTTP tape polling operations must not assume order is preserved in server response - [DMC-1323] - Gfal2 should propagate the EVICT event - [DMC-1324] - Gfal2 support for extended attributes via HTTP ###Improvement - [DMC-1260] - Expose Gfal2 XRootD eviction via the file API - [DMC-1317] - Prevent HTTP Copy fallback for non-retryable errors - [DMC-1318] - Refactor the HTTP Copy error reporting ## Fri Mar 04 2022 Mihai Patrascoiu <mipatras at cern.ch> - 2.20.5 ###Bug - [gfal2/pull/42] - Check _DARWIN_FEATURE_ONLY_64_BIT_INODE to avoid errors from stat64 being undefined - [gfal2/pull/43] - Fix building against OpenLDAP 2.5+ ###Improvement - [DMC-1308] - Move Data Management Clients to Fedora 34 & 35 - [cern-fts/gfal2/pull/9] - Remove trailing white spaces ## Tue Feb 22 2022 Mihai Patrascoiu <mipatras at cern.ch> - 2.20.4 ###Bug - [DMC-1297] - HTTP MOVE operation + SETokens needs to request a token for common base directory - [DMC-1298] - GridFTP unsupported extended attribute error message should be clearer - [DMC-1300] - Memory leak in Macaroon retrieval - [gfal2/pull/38] - Do not hardcode error numbers - [gfal2/pull/40] - Define ENODATA if not defined - [cern-fts/gfal2/pull/10] - [CMake] Explicitly state the dependency relation of the XRootD plugin and the uuid ###Request - [DMC-1296] - Improve GridFTP DNS gateway resolution mechanism ## Wed Jan 19 2022 Joao Lopes <batistal at cern.ch> - 2.20.3 ###Improvement - [gfal2/pull/41] - Simplified tokens management for CS3 storages ## Mon Dec 06 2021 Mihai Patrascoiu <mipatras at cern.ch> - 2.20.2 ###Bug - [DMC-1149] - Misleading error message from mkdir -p over GridFTP when x509 credential can not be found - [DMC-1282] - Gfal2 MKCOL with SE-tokens incompatible with dCache storage - [DMC-1283] - HTTP Rmdir + SETokens fails on DPM when path missing trailing '/' - [DMC-1284] - Do not use Authorization Header when requesting SEToken ## Mon Oct 25 2021 Mihai Patrascoiu <mipatras at cern.ch> - 2.20.1 ###Bug - [DMC-1272] - Segfault when macaroon request exceeds maximum response size - [DMC-1273] - Gfal2 HTTP plugin does not clear SE-issued tokens from the Gfal2 core credential map - [DMC-1275] - Gfal2 credential map should return partial prefix matches only in the case of directories - [DMC-1276] - Gfal2 HTTP SE-issued write token should also include all read permissions - [gfal2/pull/37] - Define ECOMM and ECHRNG if required (e.g. macOS) ###Improvement - [DMC-1274] - Provide credential delete method in the Gfal2 credential API ## Mon Sep 20 2021 Mihai Patrascoiu <mipatras at cern.ch> - 2.20.0 ###Epic - [DMC-1249] - Migrate CI infrastructure to gitlab-CI ###Bug - [DMC-1262] - Gfal2 list polling functions return error array with incomplete first element when plugin not found - [DMC-1266] - Missing null byte in readlink Gfal2 file plugin ###New Feature - [DMC-1228] - Implement SE token retrieval in Gfal2 - [DMC-1247] - Add SWIFT support in Gfal2 - [DMC-1269] - Add CS3API support in Gfal2 ###Task - [DMC-1250] - Gfal2 packages for Centos8 - [DMC-1257] - Treat "davs" and "https" protocols as the same in SRM TURL selection - [DMC-1264] - Gfal2 packages for Fedora >= 33 ###Improvement - [DMC-1246] - Avoid exposing sensitive information in Gfal2 HTTP debug output - [DMC-1261] - Improve XRootd Status code to errno mapping - [DMC-1270] - Support for checksum and getxattr calls in in Gfal2 mock plugin ## Wed May 26 2021 Mihai Patrascoiu <mipatras at cern.ch> - 2.19.2 ###Bug - [DMC-1248] - Misleading error message when XRootD bringonline bulk request fails - [DMC-1252] - Memory leak in XRootd bringonline/archive polling ## Thu Mar 25 2021 Mihai Patrascoiu <mipatras at cern.ch> - 2.19.1 ###Bug - [DMC-1237] - gfal-rm -r fails on HTTP endpoints - [DMC-1244] - XRootd copy failing when checksum value has uppercase characters ###Improvement - [DMC-1194] - gfal-xattr should avoid triggering an srmPrepareToGet in case the file is not ONLINE - [DMC-1233] - UTF8 escape XRootd error messages ## Fri Dec 04 2020 Mihai Patrascoiu <mipatras at cern.ch> - 2.19.0 ###Bug - [DMC-1227] - gfal-ls with https/davs - [DMC-1231] - Do not use BEARER tokens when dealing with S3/GCloud endpoints ###Task - [DMC-1217] - Implement gfal2 archived poll API using Xrdfs query prepare ###Improvement - [DMC-1224] - Bringonline polling should always propagate the error_text field, if it exists - [DMC-1229] - Introduce Gfal2 transfer parameter to enable/disable proxy delegation - Refactor the way S3 request parameters are obtained from the loaded configuration - Non-UTF8 characters in Davix error and log messages are correctly handled in the HTTP plugin (attributed to Petr Vokac) ## Mon Jul 27 2020 Mihai Patrascoiu <mipatras at cern.ch> - 2.18.2 ###Bug - [DMC-1220] - Filesize of GridFTP incomplete files reported as -1 ###Task - [DMC-1216] - Modify gfal2 Xrdfs query prepare to recognize both path_exists and exists fields ## Wed Jun 17 2020 Mihai Patrascoiu <mipatras at cern.ch> - 2.18.1 ###Bug - (DMC-1110) Set RequireChecksumVerification header to false also for checksum mode "both" - Fix JSON headers for HTTP plugin ## Fri Jun 05 2020 Mihai Patrascoiu <mipatras at cern.ch> - 2.18.0 ###Bug - [DMC-1201] - gfal2-plugin-xrootd incompatible with the xrootd master (v5) ###Task - [DMC-1059] - Extend the gfal2 HTTP plugin to implement CDMI QoS operations ## Thu May 07 2020 Mihai Patrascoiu <mipatras at cern.ch> - 2.17.3 ###Bug - [DMC-1203] - gfal attempts delegation with S3 / gcloud during third party copy - [DMC-1210] - gfal2 recursive mkdir fails with file exist - Fix incorrect parenthesis in GridFTP checksum validation - Token credentials support for same-host HTTP TPC transfers * Thank you to Petr Vokac for same-host HTTP TPC support using token credentials ## Tue Jan 28 2020 Michal Simon <msimon at cern.ch> - 2.17.2 ###Bug - [DMC-1198] - bring_online_poll_list: collapse redundant slashes in the path. ## Thu Nov 28 2019 Andrea Manzi <amanzi at cern.ch> - 2.17.1 ###Task - [DMC-1192] - Rewrite gfal_xrootd_bring_online_poll_list to use the Xrdfs Query Prepare implementation - [DMC-1196] - Set IPV6=false by default for gridftp plugin ###Improvement - [DMC-1187] - Add COPY_FAIL_NEARLINE option for SRM copy_file ## Tue Oct 22 2019 Andrea Manzi <amanzi at cern.ch> - 2.17.0 ###Sub-task - [DMC-1168] - add root and https to TURL_3RD_PARTY_PROTOCOLS ###Bug - [DMC-1185] - When an Xrootd staging request part of a bulk fails, all staging requests are also set as failed ###Task - [DMC-1151] - Set IPV6=true by default for gridftp plugin - [DMC-1167] - Make sure SRM<->http and SRM<->root TPC is supported - [DMC-1176] - Stop building lfc and rfio plugins on Rawhide/F31/EPEL8 ###Improvement - [DMC-1061] - xrootd plugin: checksum logic broken - [DMC-1182] - Add multistream support in Xrootd plugin ## Wed Sep 18 2019 Andrea Manzi <amanzi at cern.ch> - 2.16.4 ###Bug - [DMC-1173] - The xrootd plugin ignores the make parent option ###Task - [DMC-1159] - Implement Xrootd prepare evict on sources when Xrootd TPC transfers succeed - [DMC-1160] - Remove conditional compilation code to still support xrootd v3 on xrootd plugin - [DMC-1166] - Add support for canceling HTTP TPC transfers ## Fri May 17 2019 Andrea Manzi <amanzi at cern.ch> - 2.16.3 ###Bug - [DMC-1026] - gfal-copy segmentation fault on MacOSx - [DMC-1137] - gfal2.Gfal2Context.cancel issue for Xrootd transfers - [DMC-1147] - The autoCleanFileCopy function in gridftp plugin never executes the file unlink - [DMC-1155] - gfal-rm with signed https URL gives 401 error ###Task - [DMC-1152] - Map stat file flags for Xrootd to SRM user.status xattr - [DMC-1156] - Sanitize gridftp checksum returned by globus - [DMC-1157] - Integrate new API for staging canceling in Xrootd ## Thu Feb 21 2019 Andrea Manzi <amanzi at cern.ch> - 2.16.2 ###Bug - [DMC-1131] - Transfers gridftp->ftp fail as GLOBUS_FTP_CONTROL_MODE_EXTENDED_BLOCK is not supported on FTP ###Task - [DMC-1123] - Add ENABLE_FALLBACK_COPY variable to http plugin - [DMC-1125] - Add X509 delegation to Xrootd TPC job properties - [DMC-1129] - Add a query parameter to HTTP urls for the configuration of the copy mode ## Wed Oct 31 2018 Andrea Manzi <amanzi at cern.ch> - 2.16.1 ###Bug - [DMC-1099] - Checksum comparison in Xrootd is wrongly enabled for FTS transfers - [DMC-1106] - HTTP to HTTP streamed transfers timeout after 300 secs - [DMC-1107] - The instant throughut for HTTP Streamed transfers is wrongly calculated - [DMC-1109] - HTTP TPC failures are not logged at INFO level - [DMC-1110] - RequireChecksumVerification set to false only for HTTP TPC PULL - [DMC-1112] - Setting BEARER_TOKEN breaks pre-signed URLa ###Task - [DMC-1095] - Refactor http streamed copy to support s3 multipart upload via Davix ## Mon Seo 28 2018 Andrea Manzi <amanzi at cern.ch> - 2.16.0 ### Bug - [DMC-1048] - Xrootd plugin: the query for the EOS-CTA extended attribute "sys.retrieve.error" should be asynchronous - [DMC-1055] - The Xrootd checksum preset should not contain leading 0s - [DMC-1057] - xattr.h is in glibc-headers >= 2.27 - [DMC-1066] - gfal-sum does not honour HTTP 202 responses - [DMC-1067] - Xrootd plugin does not pass the checksum type - [DMC-1068] - Remove file after error in HTTP COPY - [DMC-1069] - Work with endpoints that support HTTP macaroons but not for COPY - [DMC-1084] - TPC with destination/source http are not working with dCache ### Task - [DMC-1046] - add support for gCloud in gfal2 http plugin - [DMC-1060] - xrootd plugin should clean destination on failure - [DMC-1064] - Add possibility to pass a BEARER token via ENV var ### Improvement - [DMC-1028] - RD_NB_STREAM in /etc/gfal2.d/gsiftp_plugin.conf has no effect - [DMC-1074] - Add DEFAULT_COPY_MODE to gfal2 http conf ## Mon May 28 2018 Andrea Manzi <amanzi at cern.ch> - 2.15.5 ### Bug - [DMC-1023] - Xrootd plugin:The checksum string provided by the user is not correctly parsed - [DMC-1041] - Incorrect GridFTP listing output ### Improvement - [DMC-1036] - gfal_http_get_token should read the credentials associated to the host ### New Feature - [DMC-1038] - Implement abort files for Xrootd bringonline - [DMC-1040] - Xrootd plugin: implement the query for the EOS-CTA extended attribute 'sys.retrieve.error' ## Thu Apr 05 2018 Andrea Manzi <amanzi at cern.ch> - 2.15.4 ### Bug - [DMC-1027] - DAVIX operation timeout incorrect for HTTP-based transfers - [DMC-1031] - gfal_http_get_token does not free the gchar* token - [DMC-1034] - When ENABLE_STREAM_COPY is disabled the http copy process crashes ## Mon March 12 2018 Andrea Manzi <amanzi at cern.ch> - 2.15.3 ### Bug - [DMC-1017] - gfal2 examples don't compile ### Improvement - [DMS-1014] - change the default copy method to PULL for HTTP - [DMC-1015] - gfal_http_copy should try all available COPY modes regardless of the error ## Mon Feb 12 2018 Alejandro Alvarez <aalvarez at cern.ch> - 2.15.2 ### Bug - [DMC-1009] - Segfault on gfal_load_configuration_to_conf_manager when there is an error loading the configuration file - [DMC-1010] - HTTP Plugin: Normalisation of destination url breaks S3 support for push third party copy - [DMC-1011] - HTTP transfers wrongly reported with success ## Thu Feb 01 2018 Alejandro Alvarez <aalvarez at cern.ch> - 2.15.1 ### Bug - [DMC-998] - XRootD plugin: xrootd is not recognised as a valid prefix for copies - [DMC-999] - GridFTP Plugin: Third party copy does not work for ftp => gridftp - [DMC-1000] - GridFTP Plugin: Disable multistreams for plain FTP - [DMC-1003] - Fix typo UNKNOW => UNKNOWN for replica locality - [DMC-1004] - SRM Plugin: Map "None" locality - [DMC-1006] - if GFAL_CRED_X509_CERT is not defined, the XROOTD PLUGIN options are skipped - [DMC-1007] - Wrong default values in configuration files (TRUE instead of true) ## Thu Nov 09 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.15.0 ### Bug - [DMC-977] - Missing closing bracket on the space reporting output - [DMC-979] - Total size reported incorrectly to be -1 - [DMC-991] - GridFTP Plugin: Reset credentials when reusing handle with a different endpoint ### New Feature - [DMC-989] - HTTP Plugin: If checksum is MD5, send Content-MD5 header ### Improvement - [DMC-954] - Drop davs+3rd, use only davs - [DMC-965] - When copying into/from srm, if the other pair is *not* srm, it should appear first on the list of protocols - [DMC-970] - Add Castor to the test endpoints - [DMC-975] - XRootD Plugin: Make normalization configurable (disable by default?) - [DMC-986] - Performance markers timeout should return ETIMEDOUT instead of ECANCEL ## Wed Jul 26 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.14.2 ### Bug - [DMC-976] - DMC-878 breaks GridFTP towards Castor ## Fri Jul 14 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.14.1 ### Bug - [DMC-973] - SRM Plugin: Revert multiple slash addition ## Wed Jun 21 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.14.0 ### Bug - [DMC-918] - LFC Plugin: Doesn't work with [X509] CERT and KEY configuration - [DMC-921] - GridFTP plugin: When asking for server, there is a missing error check - [DMC-923] - GridFTP Plugin: PASV plugin leaks - [DMC-949] - SRM Plugin: Use double slash for absolute paths - [DMC-955] - HTTP Plugin: For third party copies, fallback should be done for EPERM ### New Feature - [DMC-878] - Allow to use different credentials for different storages/paths - [DMC-931] - GridFTP Plugin: Striped Passive when IPv6 is used - [DMC-932] - XRootD Plugin: Prototype staging implementation using XRootD - [DMC-948] - LFC Plugin: Add checksum support - [DMC-953] - Copy event should specify the copy mode - [DMC-959] - Mock Plugin: Add signals on creation time, allow to disable by configuration ### Task - [DMC-929] - XRootD Plugin: Allow to specify custom xrd.* flags ### Improvement - [DMC-915] - Reorganize core - [DMC-928] - Clean up gfal2-util timeout info - [DMC-962] - Add a method to remove a configuration key ### Question - [DMC-936] - Copyright clarification ## Mon May 29 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.13.4 ### Bug - [DMC-951] - Default checksum for local copy was removed ## Fri Mar 17 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.13.3 ### Bug - [DMC-924] - gfal2 breaks backwards compatibility for checksums ## Mon Feb 20 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.13.1 ### Bug - [DMC-919] - Remove calls to globus_module_deactivate ## Mon Feb 06 2017 Alejandro Alvarez <aalvarez at cern.ch> - 2.13.0 ### Bug - [DMC-895] - Recursive directory creation: EEXIST shouldn't cause a failure - [DMC-901] - Xrootd succeeds to copy even if one of the storages does not support checksums - [DMC-909] - SRM Plugin: When BDII is disabled, SRM endpoint is built incorrectly - [DMC-910] - Leaks in BDII bindings and GridFTP plugin ### New Feature - [DMC-885] - XrootD Plugin: Add space reporting to the API - [DMC-886] - Consolidate space reporting between plugins - [DMC-892] - Generalize checksum methods - [DMC-896] - SFTP plugin ### Task - [DMC-902] - Delete old deprecated methods - [DMC-905] - Expose Low level parameters for local -> remote copies ### Improvement - [DMC-880] - If the option -t is passed, make sure that Davix knows it - [DMC-884] - XRootD Plugin: Pass desired checksum type to the storage - [DMC-900] - Apply QA tools feedback - [DMC-913] - Log a warning and keep going if loading a plugin fails ## Thu Oct 27 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.12.3 ### Bug - [DMC-891] - GridFTP Plugin: IPv6 detection misfire in some cases ## Tue Sep 06 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.12.2 ### Bug - [DMC-879] - XRootD Plugin: Segfault on 3rd party copy ## Tue Aug 16 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.12.1 ### Bug - [DMC-872] - SRM Plugin: DMC-860 broke directory listing when endpoint can not be resolved from BDII ### New Feature - [DMC-870] - Add support for s3 alternate syntax URLs to gfal ## Mon Aug 01 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.12.0 ### Bug - [DMC-833] - gfal2_add_client_info should overwrite existing key - [DMC-841] - Make scheme handling consistent across gfal2 - [DMC-846] - File Plugin: Only strict URIs must be accepted - [DMC-848] - GridFTP Plugin: PASV plugin parses incorrectly some IPv6 - [DMC-853] - [Known Issue] Connection issues when a PUT and a GET are done through the same GridFTP connection - [DMC-856] - Set user and password when accessing plain FTP - [DMC-858] - XRootD: Urldecode path before passing to xrootd libraries - [DMC-860] - SRM Plugin: Urldecode path before passing to srm-ifce ### New Feature - [DMC-827] - GridFTP Plugin: Allow to configure blocksize - [DMC-849] - GridFTP Plugin: Fire an event if IPv6 is used - [DMC-866] - Expand '-' to stdout when used as destination for gfal-copy - [DMC-868] - GridFTP Plugin: Add spacetoken xattr ### Task - [DMC-852] - GridFTP: Workaround for Globus' higher priority to /etc/grid-security/hostcert.pem ### Improvement - [DMC-837] - When an operation is not supported, include the url on the error - [DMC-851] - Improve URI parser (breaks internal API) - [DMC-855] - Accept xroot:// as well as root:// - [DMC-857] - GridFTP Plugin: Set ALLO to remote storages - [DMC-864] - Can gfal2 indicate the origin of errors (source/destination) when using GridFTP? ## Thu Apr 14 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.11.1 ### Bug - [DMC-813] - XRootD plugin: gfal2_set_error called without format string - [DMC-818] - SRM plugin not invoked when using a relative path to the source file ## Mon Mar 07 2016 Alejandro Alvarez <aalvarez at cern.ch> - 2.11.0 ### Bug - [DMC-758] - Add a Requires on glib2 >= 2.28 - [DMC-769] - GridFTP Plugin: Missing throw inside copy - [DMC-783] - SRM endpoint resolution fails due to an off-by-one introduced when switched to g_strlcpy - [DMC-798] - FIle Plugin: Readlink implemented but not exposed - [DMC-803] - GridFTP Plugin: PASV plugin fails to extract IP - [DMC-808] - gfal2_parse_uri does not support IPv6 URLS - [DMC-811] - gfal2 error GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory ### Improvement - [DMC-793] - SRM Plugin: Set "GRIDFTP PLUGIN:STAT_ON_OPEN" to false when the endpoint is Castor - [DMC-795] - XRootD Plugin: Generate better error messages - [DMC-809] - XRootD Plugin: Copy from/to XRootD using the Client (as with 3rd party copy) - [DMC-812] - Add open/read/write/close to the mock plugin ### New Feature - [DMC-756] - Mock plugin: Add directory listing and signal raising - [DMC-763] - Add plain FTP support (anonymous only) - [DMC-781] - Add support for short-term S3 credentials - [DMC-797] - HTTP Plugin: Add rename support - [DMC-801] - dCache davs pull copy use checksum verification by default, should be disabled if checksums are not asked for - [DMC-804] - Allow to pass arbitrary headers to davix ## Wed Dec 09 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.10.3 ### Bug - [DMC-783] - SRM endpoint resolution fails due to an off-by-one introduced when switched to g_strlcpy ## Fri Nov 06 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.10.2 ### Bug - [DMC-769] - GridFTP Plugin: Missing throw inside copy ## Tue Oct 20 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.10.1 ### Bug - [DMC-750] - Respect GLOBUS_THREAD_MODEL environment ### Improvement - [DMC-738] - HTTP Plugin: Skip source validation checksum if the algorithm is not supported ## Mon Oct 12 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.10.0 ### Bug - [DMC-676] - SRM Plugin: Response for bringonline may be shorter than the request - [DMC-682] - SURLS in responses may not contain full endpoint - [DMC-692] - gfal_rw_seq_SRM_DPM hangs somewhere when the proxy is not initialized - [DMC-694] - GridFTP Plugin: Malformed IPv6 printed on the logs - [DMC-717] - Remove Boost dependency from gfal2 - [DMC-720] - SRM Plugin should handle transfers also for file:/path - [DMC-722] - LFC Plugin registers the TURL rather than the SURL into the catalog - [DMC-724] - XROOTD plugin: Case-insensitive checksum type comparison - [DMC-731] - Segfault on xrootd 3rd party copies - [DMC-741] - Replace G_LOG_LEVEL_INFO with other levels - [DMC-742] - SRM Plugin: Convert UTC to local timestamp ### Improvement - [DMC-674] - Validate version as part of the packaging - [DMC-679] - SRM Plugin: Reduce log level for the unlinking messages - [DMC-685] - All: Mark in a common way when a checksum comparison fails - [DMC-698] - Port DMC clients to MacOSX - [DMC-737] - File plugin: Avoid large allocation on the stack - [DMC-744] - GridFTP Plugin: Parse uid and gid if available ### New Feature - [DMC-669] - GridFTP Plugin: Hook a plugin to capture final endpoints on transfer - [DMC-673] - SRM Plugin: Add srm.type extended attribute - [DMC-697] - HTTP Plugin: Add support for pull copies ### Task - [DMC-680] - Integrate gfal2-plugin-xrootd into mainline gfal2 - [DMC-712] - Improve mock plugin ## Tue Jun 08 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.9.3 ### Bug - [DMC-682] - SRM Plugin: SURLS in responses may not contain full endpoint - [DMC-676] - SRM Plugin: Response for poll may be shorter too ## Mon Jun 01 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.9.2 ### Bug - [DMC-676] - SRM Plugin: Response for bringonline may be shorter than the request ## Fri Apr 10 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.9.1 ### Bug - [DMC-620] - HTTP Plugin: Normalize S3:host to S3:HOST on all cases - [DMC-622] - Core: gfal2 crash when aborting staging operation - [DMC-626] - LFC Plugin: Off by one bug when checking for parent directory - [DMC-627] - LFC Plugin: When lfn:// is used, lfc_host was being left to NULL - [DMC-632] - GridFTP Plugin: When the timeout expires, wait for the callback after cancellation - [DMC-638] - GridFTP Plugin: Performance marker timeout causes abort - [DMC-639] - SRM Plugin: Fix build in Rawhide - [DMC-645] - GridFTP Plugin: Always register performance marker callback - [DMC-647] - GridFTP Plugin: In some machines, the threading model is event, and writing to castor do not work - [DMC-653] - SRM Plugin: Closing when reading must issue a release - [DMC-661] - GridFTP Plugin: Error messages generated during transfer seem to be truncated - [DMC-662] - GridFTP Plugin: Map correctly "Disk quota" errors ### Improvement - [DMC-613] - HTTP Plugin: Davix already stats before deletion, so remove the check - [DMC-617] - HTTP Plugin: Set Davix retrials to 0 - [DMC-619] - HTTP Plugin: Add KEEP_ALIVE option - [DMC-640] - Conditionally enable C++11 - [DMC-648] - Core: Provide gfal2_ prefixed methods for uri methods - [DMC-649] - Core: Improve logger system, deprecate old methods - [DMC-650] - SRM Plugin: Log the request tokens - [DMC-651] - Core and plugins: Allow to send custom key/value to the servers ### New Feature - [DMC-641] - Core: Allow to register multiple event listeners - [DMC-642] - Core: Give plugins a chance to register their own event listeners - [DMC-643] - Core: Add an event for copies that is triggered at the beginning - [DMC-657] - SRM Plugin: On prepare to get and put, get the desired request lifetime from the configuration file - [DMC-663] - Add functionality to allow querying the versions of the plugins loaded ## Fri Feb 27 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.8.4 - [DMC-638] - GridFTP Plugin: Performance marker timeout causes abort ## Tue Feb 10 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.8.3 - [DMC-632] - GridFTP Plugin: When the timeout expires, wait for the callback after cancellation ## Thu Jan 29 2014 Alejandro Alvarez <aalvarez at cern.ch> - 2.8.2 - [DMC-622] - Core: gfal2 crash when aborting staging operation ## Tue Jan 06 2015 Alejandro Alvarez <aalvarez at cern.ch> - 2.8.1 - [DMC-529] - [EPEL7 Only] GridFTP fails to load the configured credentials - [DMC-531] - GridFTP Plugin: Partial read and write missing EOF's - [DMC-541] - GridFTP and SRM plugins: Transferring from Castor SRM to non-SRM fail with "Connection refused" - [DMC-544] - GridFTP Plugin: Map "System error in name: Is a directory" to EISDIR - [DMC-553] - SRM Plugin: TURLS retrieved from a SRM endpoint should be of one of the requested protocols - [DMC-559] - SRM Plugin: gfal_srm_readdir_pipeline ignore errors - [DMC-563] - SRM Plugin: Typo in TURL resolution when copying - [DMC-566] - Mock plugin: Only size should be enough to specify the file size - [DMC-570] - GridFTP Plugin: Establishing an lfc connection before loading GridFTP makes the GridFTP plugin freeze in most (all?) operations - [DMC-580] - Core: Do not try to create parent on "transfer only" copies - [DMC-582] - SRM Plugin: Missing parameter in call to gfal2_set_error when the protocol is not expected - [DMC-584] - SRM Plugin: Cache does not invalidate entries when removing a directory - [DMC-587] - File: CRC32 should be represented in decimal - [DMC-596] - SRM Plugin: Getting file:// is a legitimate use case - [DMC-597] - GridFTP Plugin: Sessions are not reused - [DMC-598] - GridFTP Plugin: Transfer timeout is not taken into account - [DMC-601] - Core: Streamed copy buffer should be moved to the heap - [DMC-565] - Mock plugin: Reduce default sleep time for transferring to ~5 secs - [DMC-530] - SRM plugin doesn't seem to handle urls without path - [DMC-533] - GridFTP Plugin: Partial read and write, avoid seeking if not strictly required - [DMC-540] - HTTP Plugin: Assume DAV when http/https urls are used - [DMC-556] - SRM Plugin: On copies, an error on RELEASE should not be considered an error - [DMC-558] - Core: Log when events are triggered - [DMC-560] - Core: Notify events also in streamed copies - [DMC-568] - GridFTP Plugin: Enforce thread model to pthread - [DMC-574] - File Plugin: Package separately - [DMC-575] - SRM Plugin: Try listing in chunks when the storage replies "too many results" - [DMC-589] - HTTP Plugin: Reduce the verbosity level configured for Davix - [DMC-592] - GridFTP Plugin: In bulk copies, avoid redundant parent creation - [DMC-562] - SRM Plugin: Trigger events for TURL resolution when copying - [DMC-583] - Core: Add call to recover list of installed plugins - [DMC-487] - GridFTP Plugin: Refactoring - [DMC-536] - SRM Plugin: Protect the srm context with recursive mutexes - [DMC-550] - HTTP Plugin: Add option to configure davix log level for debugging - [DMC-576] - Merge gfal2, gfal2-core and gfal2-transfer in one single package - [DMC-603] - Core: Transfer logic from c++ to c - [DMC-604] - Remove dependency on glibmm24 * Mon Nov 17 2014 Alejandro Alvarez <aalvarez at cern.ch> - 2.7.8 - [DMC-553] - TURLS retrieved from a SRM endpoint should be of one of the requested protocols ## Mon Nov 10 2014 Alejandro Alvarez <aalvarez at cern.ch> - 2.7.7 - [DMC-539] - GridFTP Plugin: Transfers from gridftp endpoints hang when a read operation fails - [DMC-541] - GridFTP and SRM plugins: Transferring from Castor SRM to non-SRM fail with "Connection refused" ## Fri Nov 07 2014 Alejandro Alvarez <aalvarez at cern.ch> - 2.7.6 - [DMC-469] - LFC Plugin: Calling gfal2_set_error with wrong parameters - [DMC-473] - HTTP Plugin: 3rd party copy should translate non canonical urls (davs+3rd...) to https - [DMC-485] - Core: Recursive mkdir() fail when because of a race an intermediate directory is created - [DMC-502] - GridFTP Plugin: if ucert == ukey, do not concatenate them - [DMC-504] - SRM Plugin: mkdir-ing an existing directory must nor fail - [DMC-509] - Core: Cannot use unix special files as a destination for file transfer - [DMC-510] - srm-ifce is not mapping correctly status codes (Was Revert DMC-433) - [DMC-515] - bring online timeout is not set correctly in gfal_srmv2_bring_online_internal - [DMC-516] - SRM Plugin: Bringonline internals only checks for the status of the first file - [DMC-523] - Core: Do not try to create parent directory if it exists already (local copy) - [DMC-531] - GridFTP Plugin: Partial read and write missing EOF's - [DMC-453] - Deprecate gfal_handle and gfal_context_t inside gfal2 - [DMC-463] - GridFTP Plugin: Log storage IPv6 - [DMC-465] - HTTP Plugin: Make possible dissabling certificate verification - [DMC-507] - Core: Expose URI API publicly - [DMC-508] - Deprecate direct inclusion of header files - [DMC-533] - GridFTP Plugin: Partial read and write, avoid seeking if not strictly required - [DMC-445] - HTTP Plugin: 3rd party copies explicit (davs+3rd://) - [DMC-448] - SRM Plugin: Disable GridFTP session reuse if a castor srm endpoint is involved - [DMC-452] - Core: For convenience, xattr user.checksum directly mapped to gfal2_checksum - [DMC-459] - Core: New method to load configuration from a file (gfal2_load_opts_from_file) - [DMC-466] - LFC Plugin: Handle replica deletion via extended attributes - [DMC-490] - HTTP Plugin: Add support for AWS credentials - [DMC-493] - Core: Add gfal2_get_opt_string_with_default - [DMC-460] - GridFTP Plugin: Trimming bug in gridftp_readdir_parser - [DMC-462] - GridFTP Plugin: Pipeline copy support ## Mon Jul 28 2014 Alejandro Alvarez <aalvarez at cern.ch> - 2.6.8 - [LCGUTIL-169] - GFAL 2.0 : Enable tests for xrootd and http supports - [LCGUTIL-345] - GFAL 2.0: add support for dCache test bed for functional tests - [LCGUTIL-45] - GFAL 2.0 : non third party transfer should respect all transfers options - [LCGUTIL-166] - GFAL 2 : Update gfal 2.0 functional tests - [LCGUTIL-348] - gfal-copy produces misleading error - [LCGUTIL-355] - HTTP plugin does not do 3rd party copy when dav/davs is used as scheme - [LCGUTIL-359] - GFAL 2.0 : GFAL2 copy does not manage properly spacetoken in file <-> SRM transfer - [LCGUTIL-361] - Some transfers finish with GridFTP transferring 0 bytes, and keep going - [LCGUTIL-363] - Some tests do not clean after themselves - [LCGUTIL-365] - SRM Checksum fallback fails for IPv6 - [LCGUTIL-393] - Spacetoken operations do not handle null values - [LCGUTIL-420] - Unhandled exception causes abort - [LCGUTIL-422] - Memory leak in gfal_srm_cache_stat_add - [LCGUTIL-428] - Bringonline does not support destination space token - [LCGUTIL-429] - thread-safety issue with SSL - [LCGUTIL-432] - gfal_srmv2_bring_online returns > 0 if the file is already pinned - [LCGUTIL-433] - gfal2-srm: If the file is pinned (22), gfal_srmv2_bring_online_internal must return 1 too - [LCGUTIL-358] - When rolling back a SRM PUT, it may be necessary to explicitly remove the file - [LCGUTIL-362] - Clarify error messages coming from gfal2 on copies - [LCGUTIL-385] - GridFTP listing improvement (RELNOTE: session reuse enabled by default!) - [LCGUTIL-386] - Improvements to GridFTP 3rd party copy to achieve session reuse - [LCGUTIL-395] - Do not fail on replica registration if the same replica is already there - [LCGUTIL-412] - Delegate the grid authentication of GFAL 2.0 http plugin to davix - [LCGUTIL-435] - SRM: Cache file locality - [LCGUTIL-55] - GFAL 2.0 : set a clean API to set/get at runtime the different possible credential - [LCGUTIL-306] - lcg-stmd equivalent - [LCGUTIL-352] - support GridFTP UDT in gfal2 - [LCGUTIL-356] - Add an option to disable http 3rd party copies - [LCGUTIL-360] - Copy to http(s) does not work if the file is bigger than the buffer - [LCGUTIL-423] - Bulk bring online - [LCGUTIL-424] - Bulk deletions - [LCGUTIL-425] - srm-ifce KeepAlive is not honored - [LCGUTIL-439] - Missing abort function for gfal2 - [LCGUTIL-440] - stop perf markers callback when transferred_bytes == filesize - [LCGUTIL-326] - GFAL 2.0: Map Davix Checksum support to GFAL 2.0 - [LCGUTIL-347] - gfla2 lfc plugin: set serrno to 0 before readdir calls - [LCGUTIL-353] - HTTP plugin does not clean up destination on failure - [LCGUTIL-354] - GFAL2.0 : srm segfault problem related to GFAL 2.0 - [LCGUTIL-403] - gfal2 segfault when reading dir via srm - [LCGUTIL-406] - symbol lookup error: /usr/lib64/gfal2-plugins//libgfal_plugin_xrootd.so: undefined symbol: _ZN14XrdPosixXrootd8setDebugEi - [LCGUTIL-417] - gfal-copy return success when there is some error in the plugin - [LCGUTIL-460] - Issue with gfalFS and gridftp endpoints ## Wed Feb 26 2014 Adrien Devresse <adevress at cern.ch> - 2.5.5-1 - [LCGUTIL-40] - GFAL 2.0 : implement lfc logic that allows third party copy logic with SRM - [LCGUTIL-268] - gfal2 returns no error when proxy is invalid - [LCGUTIL-290] - Problems with dates in gfalFS - [LCGUTIL-295] - GridFTP plugin returns the full path when listing an empty directory - [LCGUTIL-284] - LFC registration (as 3rd party copy) should perform some validations - [LCGUTIL-286] - GFAL2 doesn't support multiple BDII - [LCGUTIL-287] - If LFC_HOST is not specified, it should be looked in the BDII - [LCGUTIL-288] - Implement readdirpp directly in plugins that support it - [LCGUTIL-46] - GFAL 2.0 : find a workaround for non posix file information that are not mapped to xattr - [LCGUTIL-267] - gfal2 checksum doesn't work for local files - [LCGUTIL-285] - Add gsiftp meta-data cache ## Fri Dec 06 2013 Alejandro Alvarez <aalvarez at cern.ch> - 2.4.8-0 - [LCGUTIL-216] - Checksum comparison must ignore heading '0' - [LCGUTIL-220] - gfal2 crashes with SIGSEGV sometimes on bringonline - [LCGUTIL-222] - Timeout doesn't seem to be honored - [LCGUTIL-236] - Segfault inside GridFTP_Request_state::~GridFTP_Request_state - [LCGUTIL-248] - GFAL 2.0 : Recent platform with recent glib2 version triggers "GLib-WARNING" in some case - [LCGUTIL-255] - unlink on a dir returns ERR 22 (Invalid arg) - [LCGUTIL-278] - seg fault - setxattr() in local files - [LCGUTIL-297] - gfal2 perf markers 0 must be treated as no markers and respect timeout - [LCGUTIL-160] - gfal2 needs to search the cache first (then BDII) if submitter not providing full srm url - [LCGUTIL-217] - SRM plugin should allow empty checksums on source - [LCGUTIL-303] - EOS GridFTP implementation does not return 'EEXIST' error properly - [LCGUTIL-226] - Partial listing of directory to avoid loading too much the server - [LCGUTIL-233] - GridFTP plugin should allow to skip the source checksum ## Tue Jul 02 2013 Adrien Devresse <adevress at cern.ch> - 2.3.0-0 - [LCGUTIL-99] gfal2_cancel crashes gfal2 if called twice - [LCGUTIL-135] GFAL 2.0 : Migrate the http plugin support for PEM from the old manual one to the new Davix one - [LCGUTIL-143] GFAL 2.0 : Regression bug of the stat() call against the dcache instance - [LCGUTIL-145] GridFTP plugin tries to delete the destination when getting the source checksum fails (!) - [LCGUTIL-147] GFAL 2.0 : bug on the session re-use mechanism - [LCGUTIL-165] GFAL 2.0 : enable HTTP/Webdav support by default, packaged on EPEL with davix - [LCGUTIL-179] manpage-section-mismatch - [LCGUTIL-141] GFAL 2 : add a flag in the gsiftp standard option in order to allow the gsiftp redirection, this is needed for the DPM gsiftp redirection mechanism - [LCGUTIL-51] GFAL 2.0 : implement the last missing operation : srm_mv logic for file renaming - [LCGUTIL-144] GFAL 2.0 : Update all functional test for a post-EMI usage - First version fully compatible with a standard FTS 3.0 release - Minor error correction from coverity report - bug fixes related to gridFTP thread safety for meta-data operations. ## Thu Jul 02 2013 Michail Salichos <msalicho at cern.ch> - 2.2.2-11 - pass gfal2 to coverity and fix minor issues reported ## Mon Jun 10 2013 Michail Salichos <msalicho at cern.ch> - 2.2.2-10 - fixed memory leaks and variables initialization - moved event exit message outside try-catch block - fix potential crash when can't resolve IP from hostname - LCGUTIL-142: gfal2_context_t wrapped in shared_ptr to avoid early destruction ## Thu Jun 06 2013 Michail Salichos <msalicho at cern.ch> - 2.2.2-6 - log IP and port of gridftp server in IPv4/IPv6 compatible way ## Thu May 27 2013 Michail Salichos <msalicho at cern.ch> - 2.2.2-5 - log IP and port of gridftp server ## Thu May 16 2013 Michail Salichos <msalicho at cern.ch> - 2.2.2-1 - replace gridftp exists with mlst ## Wed Mar 20 2013 Adrien Devresse <adevress at cern.ch> - 2.2.0-0 - fix thread safety issue with gsiftp plugin - add the bring online API - support for the http plugin by default - remove executable stack need - remove openMP dependency - add synchronous cancellation API - add gsiftp performance marker timeout - support for srm session reuse - reduce memory footprint ## Thu Jan 10 2013 Adrien Devresse <adevress at cern.ch> - 2.1.1-0 - fix a minor memory issue with the gfal_transfer stack - fix a wrong error report problem with srm third party copy ## Wed Dec 05 2012 Adrien Devresse <adevress at cern.ch> - 2.1.0-2 - fix an issue this surl to turl resolution for SRM third party copy ## Fri Nov 30 2012 Adrien Devresse <adevress at cern.ch> - 2.1.0-0 - One-globus session system for gsiftp plugin ( FTS 3.0 need ) - correct a major issue with the gass attribute system in gsiftp plugin - change the lfc set/get env var for a one compatible with set/get opt - add set/nb streams option for gsiftp - add the mkdir rec function for SRM transfer - correct an issue with opendir and srm_ls ( ENOTDIR error silent ) - correct a memory leak in the cache system - correct timeout support for gsiftp transfer - implement tcp buffer size support for gsiftp layer - apply a correction on the SRM over-write logic, linked to a BeStMan errcode problem on File Not Found with srmRm ( EOS ) - apply a fix on the transfer gsiftp timeout ( protection against multiple cancel ) - fix for SRM filesize problem ( defined to 0, workaround ) related to globus 426 error bad filesize - secure the callback system for globus gass timeout - base implementation of the http plugin - improve reliability of the bdii resolution - add a fallback mechanism in case of bdii bad resolution - correct several race conditions in the bdii layer - add thread safe support for set/get variables in liblfc - correct a deadlock problem with globus and gisftp plugin - implement the mkdir_rec logic for general purpose - implement the parent folder creation logic with gridftp - add support for lfc://host/path URL style for the lfc plugin - switch off_t to 64bits size by default ( _FILE_OFFSET_BITS=64) - provide a "nobdii" like option - provide the choice of turl protocol resolution for srm plugin ## Fri Jul 20 2012 Adrien Devresse <adevress at cern.ch> - 2.0.0-1 - Official initial release candidate of gfal 2.0 - Transfer API is official - gridftp support for performance marker, checksum - gridftp support for gridftpv2, dcau param - SRM support for spacetoken in transfer - SRM abort auto-management - parallel operations in transfers - file protocol dedicated in a plugin - configuration file support - srm timeout support - general purpose checksum operation support - POSIX operation support for gridftp - cleaner plugin API - new documentation - I hope that you will enjoy gfal 2.0 :) ## 2.0.0-0.10 - effective set/get opt support for lfc/srm/gsiftp - checksum calculation management - auto-check functionality on transfer - support for user-defined checksum - performance marker implementation for SRM/GSIFTP ## 2.0.0-0.9 - Without environment variable design implemented - add set/get option calls - add SRM src/dst spacetoken management for gfal transfer - re-factor properly gfal-transfer and gfal2_context_t - generate a new clean documentation with log/transfer/context support - allow direct modification of the internal posix context - fix memory leak with gsiftp error report ## 2.0.0-0.8 - big improvement on the gridftp support - split the headers files properly - remove the carriage return from the gridftp error messages - add plugin dedicaded API. - refactor the parameter management for the transfer library - add the "transfer replace" logic for the transfer library ( FTS 3.0 requirement ) - add logger system based on the glib log system ( ARC requirement ) - correct memory leaks and a session-reuse memory corruption for gridftp - improve gridFTP global speed with request optimisations. - remove the old config/parameters system for the posix library. - add several functional tests for the new features. - add SRM to X and X to SRM modes to the SRM plugin ( FTS 3.0 requirement ) ## 2.0.0-0.7 - major improvement for third party transfer - initial gridftp support - begin to switch to configuration without env var ## 2.0.0-0.6 - initial tagged release