Package org.gcube.documentstore.records.aggregation

Class AggregationScheduler

java.lang.Object
org.gcube.documentstore.records.aggregation.AggregationScheduler
All Implemented Interfaces:
Runnable
Direct Known Subclasses:
BufferAggregationScheduler
public abstract class AggregationScheduler extends Object implements Runnable
Author:
Luca Frosini (ISTI - CNR)

  • Field Details

    • logger

      public static final org.slf4j.Logger logger

    • AGGREGATION_SCHEDULER_TIME

      public static final String AGGREGATION_SCHEDULER_TIME
      See Also:

    • BUFFER_RECORD_TIME

      public static final String BUFFER_RECORD_TIME
      See Also:

    • BUFFER_RECORD_NUMBER

      public static final String BUFFER_RECORD_NUMBER
      See Also:

    • RANDOM_INIT_START

      public static final int RANDOM_INIT_START
      See Also:

    • TIME_RELOAD_CONFIGURATION

      public static final long TIME_RELOAD_CONFIGURATION
      The Max amount of time for reload a configuration Get from configuration
      See Also:

    • CONFIG_DIRECTORY_NAME

      public static final String CONFIG_DIRECTORY_NAME
      See Also:

    • FILE_NAME

      public static final String FILE_NAME
      See Also:

    • AGGREGATION_PROPERTIES_FILE

      public static final File AGGREGATION_PROPERTIES_FILE

    • totalBufferedRecords

      protected int totalBufferedRecords

    • bufferedRecords

      protected Map<String,List<Record>> bufferedRecords

    • persistenceExecutor

      protected final PersistenceExecutor persistenceExecutor

    • changeConfiguration

      public boolean changeConfiguration

    • futureFlush

      protected ScheduledFuture<?> futureFlush

    • futureReload

      protected ScheduledFuture<?> futureReload

  • Constructor Details

  • Method Details

    • newInstance

      public static AggregationScheduler newInstance(PersistenceExecutor persistenceExecutor)

    • newInstance

      public static AggregationScheduler newInstance(PersistenceExecutor persistenceExecutor, PersistenceBackendConfiguration configuration) throws NumberFormatException, Exception
      Throws:
      NumberFormatException
      Exception

    • instantiateAggregatedRecord

      protected static AggregatedRecord instantiateAggregatedRecord(Record record) throws Exception
      Throws:
      Exception

    • getAggregatedRecord

      public static AggregatedRecord getAggregatedRecord(Record record) throws Exception
      Throws:
      Exception

    • madeAggregation

      protected void madeAggregation(Record record)

    • flush

      public void flush(PersistenceExecutor persistenceExecutor) throws Exception
      Throws:
      Exception

    • schedulerSpecificClear

      protected abstract void schedulerSpecificClear()

    • clear

      protected void clear()

    • aggregate

      protected void aggregate(Record record, PersistenceExecutor persistenceExecutor, boolean forceFlush) throws Exception
      Aggregates the record, if any, and persists the buffered records when it is time to, or when forced. The buffer is taken under this scheduler's lock, and persisted outside it: a slow persistence does not block the aggregation of the incoming records.
      Parameters:
      record - the record to aggregate, or null to only check whether to persist
      persistenceExecutor - the executor persisting the records
      forceFlush - whether to persist the buffered records in any case
      Throws:
      Exception - if the persistence fails; the records are then buffered again

    • takeBufferedRecords

      protected Record[] takeBufferedRecords()
      Takes the buffered records out of the buffer.
      Returns:
      the buffered records, or null if the buffer is empty

    • persist

      protected void persist(PersistenceExecutor persistenceExecutor, Record[] recordsToPersist) throws Exception
      Persists the records, one persistence at a time for this scheduler. If the persistence fails, the records are buffered again, so that the next flush retries them.
      Parameters:
      persistenceExecutor - the executor persisting the records
      recordsToPersist - the records
      Throws:
      Exception - if the persistence fails

    • flushAfterWaiting

      public void flushAfterWaiting(PersistenceExecutor persistenceExecutor, long timeout, TimeUnit timeUnit) throws Exception
      Persists the buffered records with the given executor after waiting, at most for the given time, for the persistence in progress, if any. Meant for a local executor at shutdown, e.g. the fallback file: the persistence in progress gets a chance to complete, or to put its records back into the buffer; if it does not complete in time, its outcome is unknown and the buffered records are persisted anyway.
      Parameters:
      persistenceExecutor - the executor persisting the records
      timeout - the maximum time to wait for the persistence in progress
      timeUnit - the unit of the timeout
      Throws:
      Exception - if the persistence fails

    • reallyFlush

      protected void reallyFlush(PersistenceExecutor persistenceExecutor) throws Exception
      Throws:
      Exception

    • aggregate

      public void aggregate(Record record, PersistenceExecutor persistenceExecutor) throws Exception
      Get a record and try to aggregate with other buffered Records.
      Parameters:
      record - The Record
      persistenceExecutor -
      Throws:
      Exception

    • isTimeToPersist

      protected abstract boolean isTimeToPersist(int maxRecordNumber, long oldRecordMaxTime)

    • reloadConfiguration

      protected void reloadConfiguration()
      reloadConfiguration

    • getConfiguration

      protected PersistenceBackendConfiguration getConfiguration()
      Get Configuration (used from reload configuration)
      Returns:
      PersistenceBackendConfiguration

    • getPropertiesFromFile

      public static Properties getPropertiesFromFile() throws IOException
      Throws:
      IOException

    • CheckConfiguration

      protected static AggregationConfiguration CheckConfiguration(PersistenceBackendConfiguration configuration) throws IOException
      Throws:
      IOException

    • getConfig

      public AggregationConfiguration getConfig()

    • setConfig

      public void setConfig(AggregationConfiguration newConfig)

    • run

      public void run()
      Specified by:
      run in interface Runnable

    • shutdown

      public void shutdown()