File Downloader

File downloader utility methods and classes.

vibe_core.file_downloader.build_file_path(dir_name, file_name, type='')

Build the full file path.

This is done by combining the directory name, file name and optional type to infer the file extension.

Parameters:

  • dir_name (str) – Name of the directory.

  • file_name (str) – Name of the file.

  • type (str) – Type of the file (default is empty).

Returns:

The full file path.

Return type:

str

vibe_core.file_downloader.download_file(url, file_path, chunk_size=1048576, connect_timeout=30, read_timeout=30, **kwargs)

Download a file from a given URL to the given file path.

The download is done using a retry session, to handle connection errors.

Parameters:

  • url (str) – URL of the file to download.

  • file_path (str) – Path where the file will be saved.

  • chunk_size (int) – Amount of data to read from the server per request (defaults to CHUNK_SIZE).

  • connect_timeout (float) – Time in seconds to wait for connection to the server before aborting (defaults to CONNECT_TIMEOUT_S).

  • read_timeout (float) – Time in seconds for each chunk read from the server (defaults to READ_TIMEOUT_S).

  • kwargs (Any) – Additional keyword arguments to be passed to the request library call.

Returns:

Path of the saved file.

Return type:

str

vibe_core.file_downloader.retry_session()

Create a session with retry support.

This method creates a requests.Session object with retry support configured to retry failed requests up to REQUEST_RETRIES times with a REQUEST_BACKOFF time back-off factor.

Returns:

A configured requests.Session object

Return type:

Session

vibe_core.file_downloader.verify_url(url, connect_timeout=30, **kwargs)

Verify the validity of a given URL.

This method attempts to connect to the specified url and verifies that it does not raise any HTTP or Connection errors.

Parameters:

  • url (str) – The URL to check.

  • connect_timeout (float) – Timeout when attempting to connect to the specified url. Defaults to the value of CONNECT_TIMEOUT_S.

  • kwargs (Any) – Additional keyword arguments to pass to the requests.get call.

Returns:

True if the URL is valid, False otherwise.

Return type:

bool

vibe_core.file_downloader.CHUNK_SIZE = 1048576

Size of the chunks to read from the server per request.

vibe_core.file_downloader.CONNECT_TIMEOUT_S = 30

Time in seconds to wait for connection to the server before aborting.

vibe_core.file_downloader.READ_TIMEOUT_S = 30

Time in seconds for each chunk read from the server.

vibe_core.file_downloader.REQUEST_BACKOFF = 0.3

Back-off factor to apply between retries.

vibe_core.file_downloader.REQUEST_RETRIES = 5

Number of retries to perform when a request fails.