File Downloader

vibe_core.file_downloader.build_file_path(dir_name, file_name, type='')

Builds the full file path by combining the directory name, file name and optional type to infer the file extension.

Parameters:

  • dir_name (str) – Name of the directory.

  • file_name (str) – Name of the file.

  • type (str, default: '') – Type of the file (default is empty).

Returns:

str – The full file path.

vibe_core.file_downloader.download_file(url, file_path, chunk_size=1048576, connect_timeout=30, read_timeout=30, **kwargs)

Downloads a file from a given URL to the given file path.

The download is done using a retry session, to handle connection errors.

Parameters:

  • url (str) – URL of the file to download.

  • file_path (str) – Path where the file will be saved.

  • chunk_size (int, default: 1048576) – Amount of data to read from the server per request (defaults to CHUNK_SIZE).

  • connect_timeout (float, default: 30) – Time in seconds to wait for connection to the server before aborting (defaults to CONNECT_TIMEOUT_S).

  • read_timeout (float, default: 30) – Time in seconds for each chunk read from the server (defaults to READ_TIMEOUT_S).

  • kwargs (Any) – Additional keyword arguments to be passed to the request library call.

Returns:

str – Path of the saved file.

vibe_core.file_downloader.retry_session()

Creates a session with retry support.

This method creates a requests.Session object with retry support configured to retry failed requests up to REQUEST_RETRIES times with a REQUEST_BACKOFF time back-off factor.

Returns:

Session – A configured requests.Session object

vibe_core.file_downloader.verify_url(url, connect_timeout=30, **kwargs)

Verifies the validity of a given URL.

This method attempts to connect to the specified url and verifies that it does not raise any HTTP or Connection errors.

Parameters:

  • url (str) – The URL to check.

  • connect_timeout (float, default: 30) – Timeout when attempting to connect to the specified url. Defaults to the value of CONNECT_TIMEOUT_S.

  • kwargs (Any) – Additional keyword arguments to pass to the requests.get call.

Returns:

bool – True if the URL is valid, False otherwise.

vibe_core.file_downloader.CHUNK_SIZE = 1048576

Size of the chunks to read from the server per request.

vibe_core.file_downloader.CONNECT_TIMEOUT_S = 30

Time in seconds to wait for connection to the server before aborting.

vibe_core.file_downloader.READ_TIMEOUT_S = 30

Time in seconds for each chunk read from the server.

vibe_core.file_downloader.REQUEST_BACKOFF = 0.3

Back-off factor to apply between retries.

vibe_core.file_downloader.REQUEST_RETRIES = 5

Number of retries to perform when a request fails.