Update _read_bytes function in images.py support for URL #548
Conversation
Add support for fetching images from URLs in _read_bytes_image Previously, the function only supported reading image files from local paths. This commit extends its functionality to also fetch images from URLs if the path starts with 'http://' or 'https://'. The function now uses the 'requests' library to download the image content when a URL is provided. If the URL fetch is successful, the image content is returned as bytes. If the status code is not 200, a ValueError is raised with a message indicating the failure. The existing file reading functionality remains unchanged for non-URL paths, ensuring backward compatibility. Changes: - Modified _read_bytes_image to check if the path is a URL - Added conditional logic to handle URL paths using requests.get - Included error handling for failed URL fetch attempts
|
Hey, @stoensin ! Thank you for the interest in OML. I think we need some tests for this. As an input you can use this url: |
| return fin.read() | ||
| if isinstance(path, str) and path.startswith(('http://', 'https://')): | ||
| # If the path is a URL, use requests to fetch the content | ||
| response = requests.get(path) |
There was a problem hiding this comment.
From a performance point of view, it would be more efficient to create a session instance self.session = self.session or requests.Session() and make requests using self.session.get(...). Otherwise, for each call of __getitem__, you will need to establish a new connection with the remote server (DNS resolution, certificate exchange, etc). Establishing a connection can take a huge amount of time.
I'm not sure about an exact implementation but doing requests.get is not a good idea for dataset scenario, because we can reuse a session.
|
Hey, @stoensin ! Let's move on with this PR. Please, see the comment above: #548 (comment) |
Add support for fetching images from URLs in _read_bytes_image
Previously, the function only supported reading image files from local paths. This commit extends its functionality to also fetch images from URLs if the path starts with 'http://' or 'https://'. The function now uses the 'requests' library to download the image content when a URL is provided.
If the URL fetch is successful, the image content is returned as bytes. If the status code is not 200, a ValueError is raised with a message indicating the failure.
The existing file reading functionality remains unchanged for non-URL paths, ensuring backward compatibility.
Changes:
For now, we don't push you to follow any predefined schema of issue, but ensure you've already read our contribution guide: https://open-metric-learning.readthedocs.io/en/latest/from_readme/contributing.html.