T
- stream tuple typepublic class CompressedFileWriterPolicy<T> extends FileWriterPolicy<T>
FileWriterPolicy
that generates zip compressed files.
CompressedFileWriterPolicy
is used exactly like FileWriterPolicy
.
The generated file names are identical to those generated by FileWriterPolicy
except they have a .zip
suffix.
The active file is uncompressed.
It is compressed when cycled per the FileWriterCycleConfig
.
Hence, a file size based
cycle config specifies the size of the uncompressed active file.
An aggregate
file size
based retention config specifies the total size of the
retained compressed files.
// Create a CompressedFileWriterPolicy with the configuration:
// no explicit flush; cycle the active file when it exceeds 200Kb;
// retain up to 1Mb of compressed files.
IFileWriterPolicy<String> policy = new CompressedFileWriterPolicy(
FileWriterFlushConfig.newImplicitConfig(),
FileWriterCycleConfig.newFileSizeBasedConfig(200_000),
FileWriterRetentionConfig.newAggregateFileSizeBasedConfig(1_000_000));
String basePathname = "/some/directory/and_base_name";
TStream<String> streamToWrite = ...
FileStreams.textFileWriter(streamToWrite, () -> basePathname, () -> policy)
Constructor and Description |
---|
CompressedFileWriterPolicy() |
CompressedFileWriterPolicy(FileWriterFlushConfig<T> flushConfig,
FileWriterCycleConfig<T> cycleConfig,
FileWriterRetentionConfig retentionConfig) |
Modifier and Type | Method and Description |
---|---|
protected void |
compressFile(java.nio.file.Path src,
java.nio.file.Path dst) |
protected java.nio.file.Path |
hookGenerateFinalFilePath(java.nio.file.Path path)
Generate the final file path for the active file.
|
protected void |
hookRenameFile(java.nio.file.Path activePath,
java.nio.file.Path finalPath)
"Rename" the active file to the final path.
|
close, closeActiveFile, getCycleConfig, getFlushConfig, getNextActiveFilePath, getRetentionConfig, hookGenerateNextActiveFilePath, initialize, shouldCycle, shouldFlush, toString, wrote
public CompressedFileWriterPolicy()
public CompressedFileWriterPolicy(FileWriterFlushConfig<T> flushConfig, FileWriterCycleConfig<T> cycleConfig, FileWriterRetentionConfig retentionConfig)
protected java.nio.file.Path hookGenerateFinalFilePath(java.nio.file.Path path)
FileWriterPolicy
The default implementation yields:
final file names are basePathname_YYYYMMDD_HHMMSS[_<n>]
where the optional _<n>
suffix is only present if needed
to distinguish a file from the previously finalized file.
<n>
starts at 1 and is monitonically incremented.
This hook method can be overridden.
Note, the implementation must handle the unlikely, but happens in tests, case where files are cycling very fast (multiple per sec) and the retention config tosses some within that same second. I.e., avoid generating final path sequences like:
leaf_YYYYMMDD_103099 leaf_YYYYMMDD_103099_1 leaf_YYYYMMDD_103099_2 delete leaf_YYYYMMDD_103099 -- retention cnt was 2 leaf_YYYYMMDD_103099 // should be _3
hookGenerateFinalFilePath
in class FileWriterPolicy<T>
path
- the active file path to finalizeprotected void hookRenameFile(java.nio.file.Path activePath, java.nio.file.Path finalPath) throws java.io.IOException
FileWriterPolicy
The default implementation uses java.io.File.renameTo()
and works for the default FileWriterPolicy.hookGenerateNextActiveFilePath()
and FileWriterPolicy.hookGenerateFinalFilePath(Path path)
implementations.
This hook method can be overridden.
hookRenameFile
in class FileWriterPolicy<T>
activePath
- path of the active filefinalPath
- path to the final destinationjava.io.IOException
- on failureprotected void compressFile(java.nio.file.Path src, java.nio.file.Path dst) throws java.io.IOException
java.io.IOException
Copyright © 2016 The Apache Software Foundation. All Rights Reserved - bbe71fa-20161201-1641