Tuning FasterLog

FasterLog may be configured using the FasterLogSettings input to the FasterLog constructor.

var log = new FasterLog(new FasterLogSettings { ... });

The following log settings are available for configuration:

  • LogDevice: This is an instance of the storage interface (device/path/file) that will be used for the main log. It is a class that implements the IDevice interface. FASTER provides several device implementations out of the box, such as LocalStorageDevice and ManagedLocalStorageDevice for local (or attached) storage. You can use our extension method to easily create an instance of a local device: 
      var log = Devices.CreateLogDevice(Path.GetTempPath() + "hlog.log");

    When using FASTER in memory, you can set the device to a null device (NullDevice) by providing an empty string as parameter to the method above. You are responsible for disposing the device via IDisposable.

  • PageSizeBits: This field (P) is used to indicate the size of each page. It is provided in terms of the number of bits. You get the actual size of the page by a simple computation of two raised to the power of the number specified (2P). For example, when PageSizeBits P is set to 12, it represents pages of size 4KB (since 2P = 212 = 4096 = 4KB). Generally you should not need to adjust PageSizeBits from its default value of 25 (225 = 32MB page size). You cannot use a page size smaller than the device sector size (512 bytes by default), which translates to a PageSizeBits value of at least 9.

  • MemorySizeBits: This field (M) is used to indicate the total size of memory used by the log. As before, for a setting of M, 2M is the number of bytes used totally by the log. Since each page is of size 2P, the number of pages in memory is simply 2M-P. FASTER requires at least 1 page in memory, so MemorySizeBits should be set to at least P.

  • SegmentSizeBits: On disk, the data is written to files in coarse-grained chunks called segments. We can size each chunk independently of pages, as one segment typically consists of many pages. SegmentSizeBits is used to indicate the size of each segment. As before, it is specified in bits. For instance, if we want each file on disk to be 1GB (the default), we can set SegmentSizeBits S to 30, since 230 = 1GB.

  • MutableFraction: This field (F) indicates the fraction of the memory that is marked as mutable, i.e., uncommitted (default 0). A value of 0.5 indicates that we will only automatically flush log pages in the first 50% of in-memory pages. Note that a commit will flush all data regardless of this setting, i.e., this setting only applies to automatic log flushing without a commit.

  • LogCommitManager: An (optional) custom log commit manager provided by user to override the default one.

  • LogCommitFile: Use specified directory for storing and retrieving checkpoints. This is simply a shortcut to providing the following: FasterLogSettings.LogCommitManager = new LocalLogCommitManager(LogCommitFile);.

  • GetMemory: User callback to allocate memory for read entries (optional).

  • LogChecksum: Type of checksum to add to log entries.

  • ReadOnlyMode: Use FasterLog as a read-only iterator/viewer of log being committed by another instance, possibly on a different process or machine.

Updated: