Resources v26.0.0 2023-09-22¶
This package now requires Python 3.10 and newer.
New Features¶
resource
URI schemes now useimportlib.resources
(orimportlib_resources
) rather than the deprecatedpkg_resources
. Due to this change,resource
URI schemes now also supportwalk
andfindFileResources
. (DM-33528)Modified the way that a schemeless absolute URI behaves such that we now always convert it to a
file
URI.The
root
parameter can now use anyResourcePath
scheme such that a relative URI can be treated as a URI relative to, for example, a S3 or WebDAV root. (DM-38552)
The
LSST_DISABLE_BUCKET_VALIDATION
environment variable can now be set to disable validation of S3 bucket names, allowing Ceph multi-tenant colon-separated names to be used. (DM-38742)Added support for
as_local
for Python package resource URIs.Added explicit
isdir()
implementation for Python package resources. (DM-39044)
Bug Fixes¶
Fixed problem where a fragment associated with a schemeless URI was erroneously being quoted. (DM-35695)
Fixed invalid endpoint error in the
FileReadWriteTestCase
test when theS3_ENDPOINT_URL
environment variable is set to an invalid endpoint. (DM-37439)Fixed EOF detection with S3 and HTTP resource handles when using repeated
read()
.Ensured that HTTP reads with resource handles using byte ranges correctly disable remote compression. (DM-38589)
Reorganized
mexists()
implementation to allow S3 codepath to ensure that a client object was created before using multi-threading. (DM-40762)
Miscellaneous Changes of Minor Interest¶
ResourcePathExpression
can now be used in anisinstance
call on Python 3.10 and newer. (DM-38492)
An API Removal or Deprecation¶
Dropped support for Python 3.8 and 3.9. (DM-39791)
Resources v25.0.0 2023-02-27¶
Miscellaneous Changes of Minor Interest¶
For file copies with
transfer_from()
an attempt is now made to make the copies atomic by usingos.rename
with a temporary intermediate. Moves now explicitly preferos.rename
and will fall back to an atomic copy before deletion if needed. This is useful if multiple processes are trying to copy to the same destination file. (DM-36412)Added
allow_redirects=True
to WebDAV HEAD requests since the default isFalse
. This is needed when interacting with WebDAV storage systems which have a frontend redirecting to backend servers. (DM-36799)
Resources v24.0.0 2022-08-26¶
New Features¶
This package is now available on PyPI as lsst-resources.
The
lsst.daf.butler.ButlerURI
code has been extracted from thedaf_butler
package and made into a standalone package. It is now known aslsst.resources.ResourcePath
and distributed in thelsst-resources
package.Add support for Google Cloud Storage access using the
gs
URI scheme. (DM-27355)Builds using
setuptools
now calculate versions from the Git repository, including the use of alpha releases for those associated with weekly tags. (DM-32408)Add an
open
method that returns a file-like buffer wrapped by a context manager. (DM-32842)Major cleanup of the WebDAV interface:
Improve client timeout and retries.
Improve management of persistent connections to avoid exhausting server resources when there are thousands of simultaneous clients.
Rename environment variables previously named
LSST_BUTLER_*
by:LSST_HTTP_CACERT_BUNDLE
LSST_HTTP_AUTH_BEARER_TOKEN
LSST_HTTP_AUTH_CLIENT_CERT
LSST_HTTP_AUTH_CLIENT_KEY
LSST_HTTP_PUT_SEND_EXPECT_HEADER
(DM-33769)
Miscellaneous Changes of Minor Interest¶
Reorganize test code to enhance code reuse and allow new schemes to make use of existing tests. (DM-33394)
Attempt to catch 429 Retry client error in S3 interface. This code is not caught by
botocore
itself since it is not part of the AWS standard but Google can generate it. (DM-33597)When walking the local file system symlinks to directories are now followed. (DM-35446)