cctools
ndcctools.resource_monitor.Categories Class Reference

Class to encapsule all the categories in a workflow. More...

Public Member Functions

 __init__ (self, all_categories_name='(all)')
 Create an empty set of categories.
 
 category_names (self)
 Returns a lists of the category categories.
 
 first_allocation (self, mode, category)
 Compute and return the first allocations for the given category.
 
 maximum_seen (self, category)
 Return the maximum resource values so far seen for the given category.
 
 accumulate_summary (self, summary)
 Add the summary (a dictionary) to the respective category.
 
 waste (self, category, field, allocation)
 Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.
 
 wastepercentage (self, category, field, allocation)
 Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.
 
 throughput (self, category, field, allocation)
 Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.
 
 retries (self, category, field, allocation)
 Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.
 
 count (self, category)
 Return the number of summaries in a particular category.
 

Detailed Description

Class to encapsule all the categories in a workflow.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 60, 'cores': 1, ... } )
print(cs.first_allocation(mode = 'throughput', category = 'some_category'))

Constructor & Destructor Documentation

◆ __init__()

ndcctools.resource_monitor.Categories.__init__ ( self,
all_categories_name = '(all)' )

Create an empty set of categories.

Parameters
selfReference to the current object.
all_categories_nameName of the general category that holds all of the summaries.

Member Function Documentation

◆ category_names()

ndcctools.resource_monitor.Categories.category_names ( self)

Returns a lists of the category categories.

List sorted lexicographicaly, with the exception of self.all_categories_name, which it is always the last entry.

Parameters
selfReference to the current object.

◆ first_allocation()

ndcctools.resource_monitor.Categories.first_allocation ( self,
mode,
category )

Compute and return the first allocations for the given category.

Note: wall_time needs to be defined in the resource summaries to be considered in this optimization.

Parameters
selfReference to the current object.
modeOptimization mode. One of 'throughput', 'waste', or 'fixed'.
categoryName of the category
cs = Categories()
fa = cs.first_allocation(mode = 'throughput, category = 'some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

◆ maximum_seen()

ndcctools.resource_monitor.Categories.maximum_seen ( self,
category )

Return the maximum resource values so far seen for the given category.

Parameters
selfReference to the current object.
categoryName of the category
cs = Categories()
fa = cs.maximum_seen('some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

◆ accumulate_summary()

ndcctools.resource_monitor.Categories.accumulate_summary ( self,
summary )

Add the summary (a dictionary) to the respective category.

At least both the 'category' and 'wall_time' keys should be defined.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 50, 'cores': 1, ... } )

◆ waste()

ndcctools.resource_monitor.Categories.waste ( self,
category,
field,
allocation )

Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

◆ wastepercentage()

ndcctools.resource_monitor.Categories.wastepercentage ( self,
category,
field,
allocation )

Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

◆ throughput()

ndcctools.resource_monitor.Categories.throughput ( self,
category,
field,
allocation )

Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

◆ retries()

ndcctools.resource_monitor.Categories.retries ( self,
category,
field,
allocation )

Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

◆ count()

ndcctools.resource_monitor.Categories.count ( self,
category )

Return the number of summaries in a particular category.

Parameters
selfReference to the current object.
categoryName of the category

The documentation for this class was generated from the following file: