app_configs Class

This class serves as base class to every Happyface analysis. There should be a class inheriting from this one in each apps.py file in every analysis. Therefore this class is only abstract and most functions need to be overwritten.

Note

Some analyses define their own AppConfig subclass and do not inherit from AnalysisConfig. Currently these are the legacy modules hammercloud (still enabled, wires getData/builddiv as module functions) and the disabled site_readiness and images. New analyses should always inherit from AnalysisConfig.

class Happyface4.app_configs.AnalysisConfig(app_name, app_module)

Bases: AppConfig

Abstract class handling the common methods of an Analysis and representing all of the analysis configurations.

final builddiv(time_of_readout, instance, request)

Central function which is called if there is an HTTP request to a category with this instance in it. Calls the retrieve_data_from_db function and renders the template with the data obtained from the database. Analyses with db_data_required = False fetch all data from the client and are rendered without the database lookback.

Parameters:

• time_of_readout (datetime) – The time the data is requested for.

• instance (dict) – The dict with the properties of an analysis instance.

• request (HttpRequest) – A django request instance with all the information needed to answer the request.

Returns:

The rendered HTML code for this analysis.

Return type:

str

db_data_required: bool = True

Whether builddiv reads analysis data from the database for the initial render. Analyses that fetch all their data asynchronously from the client set this to False to skip the database lookback.

abstractmethod extract_data_from_url(instance)

This function uses the source given in the instance to fetch the data. This function is individual to each source and should therefore be overwritten.

Parameters:

instance (dict) – The dict with the properties of an analysis instance, this is passed automatically by the getData() function.

Returns:

The data fetched from the url.

Return type:

dict or list of dicts or list

final getData(instance)

Central function which is called by the global getData command for each instance. Handles the process of fetching and saving data and calculating the instance status.

Parameters:

instance (dict) – The dict with the properties of an analysis instance, this is passed by the global getData function.

get_ajax_endpoints(instance, timestamp)

Optional method for analysis modules to declare their AJAX endpoints for cache warmup.

Override this method in your analysis config to enable AJAX endpoint caching during cache warmup. This prevents empty plots/tables during getData execution.

Parameters:

• instance (dict) – The dict with the properties of an analysis instance.

• timestamp (int) – Unix timestamp to use for time_of_readout parameter.

Returns:

List of dicts, each containing: - ‘url_name’: Django URL pattern name (e.g., ‘dcache_transfer_get_data’) - ‘url_kwargs’: Dict of URL kwargs (e.g., {‘instance_name’: ‘dCache_Host’}) - ‘query_params’: Dict of query parameters (e.g., {‘time_of_readout’: 123, ‘dt’: 86400})

Returns empty list by default (no AJAX endpoints).

Return type:

list

Example implementation in dcache_transfer/apps.py:

def get_ajax_endpoints(self, instance, timestamp):
    return [
        {
            'url_name': 'dcache_transfer_get_data',
            'url_kwargs': {'instance_name': instance['name']},
            'query_params': {
                'time_of_readout': timestamp,
                'dt': instance.get('time_window', 86400)
            }
        }
    ]

get_instance_status(instance, time, time_range=None)

This function gets or calculates the status of an instance. The base implementation queries the stored instance statuses; subclasses may override it to compute the status from the data instead.

Parameters:

• instance (dict) – The dict with the properties of an analysis instance the instance status was requested for.

• time (datetime) – The time in which the data was retrieved for which the instance status is requested.

• time_range (timedelta) – Specifies the time range in which to search for statuses, i.e. statuses between time - time_range and time are considered.

Returns:

A QuerySet of InstanceStatus objects, or a single DummyInstanceStatus object when no status is available in the requested time range.

Return type:

QuerySet or DummyInstanceStatus

instance_parallelization: bool = False

Enable or disable parallel execution of instances within this analysis.

When set to True, instances will be processed in parallel using ThreadPoolExecutor, which can significantly reduce getData execution time for analyses with multiple instances.

Important: Only set this to True if you have verified that:

1. Instances do not share database tables (or use proper locking)

2. Instances do not have race conditions in save_data_to_db()

3. Each instance operates on independent data

Default is False for safety.

Example:

class QueuedDataTransfersConfig(AnalysisConfig):
    name = "queued_data_transfers"
    instance_parallelization = True  # Safe - verified no conflicts
    instances = [...]

abstract property instances: list

List of dicts which contain the info and settings of the different instances of one Analysis.

Should have at least the following contents:

instances = [
    {
    # Name of the analysis instance
    "name": "happyface4_analysis_instance_1",
    # This will get the visible header of the analysis
    "verbose_name": "Happyface 4 Analysis Example 1",
    # Category of the instance. Each category is displayed in a separate
    # site.
    "category": "Batch System",
    # Url to data source
    "source": "example.com",
    # A short description of the instance
    "description": "Status of the Compute Element Tests",
    # If the instance doesn't have an internal status to set and has only
    # the status `info`, this flag can be set to True.
    # In this case the `get_instance_status()` method doesn't need to be
    # specified and `save_data_to_db()` should return `None,None`.
    "is_info": True,
    # A threshold for the 'critical' and the 'warning' alert level. Putting
    # it here is useful if there are different threshold for different
    # instances in an analysis.
    "critical_thresholds": 0.8,
    "warning_thresholds": 0.5,
    # The file for the HTML template.
    "template": "happyface4_analysis/happyface4_analysis.html",
    # The time interval in which the data is fetched. This is an int giving multiples of settings.PULL_INTERVAL
    "get_data_every": 1,
    }
]

abstract property name: str

standard name of the analysis is the module name, which is the name of the folder in which the analysis is located

abstractmethod retrieve_data_from_db(instance, time_of_readout, dt)

This function is responsible for loading the data of this instance from the database for the desired time. The filtering of the database for the desired time can be defined in the managers.py file, so this function only needs to check if data is available at that time. Then the data must be returned together with the timestamp. The timestamp shows when the data is from. The default implementations of this abstract method only log an error and return None; it needs to be overwritten unless the analysis sets db_data_required = False and fetches all data from the client.

Parameters:

• instance (dict) – The dict with the properties of an analysis instance the data was requested for.

• time_of_readout (datetime) – The date and time the data was requested for.

• dt (timedelta) – Specifies the time range in which to search for data, i.e. data between time_of_readout - dt and time_of_readout is considered.

Returns:

• data - A dict or

  Django Queryset with the data. This is directly loaded in the template and has to be prepared for it.

• timestamp - The date and time the data is from.

Return type:

(dict or QuerySet, datetime)

abstractmethod save_data_to_db(data, instance)

This function takes the fetched data and prepares it for the database and finally saves it in the database. This function needs to be overwritten! The preparation includes removing all data not needed and making all analytical calculation needed. The more calculation is done here the less calculation needs to be done when an HTML request comes and therefore the site will load faster. But remember this function is called every time the global getData function is called (e.g. every 15min) so the calculations should not be too big.

To save the data to the database (you can define the database tables in the models.py) you need to do something like this:

from . import models
models.ExampleModel.objects.create(instance=instance['name'],             example_data=prepared_data, ...)

For more information please look into the django documentation.

Parameters:

• data (dict or list) – This is exactly the data which is returned from the extract_data_from_url() function.

• instance (dict) – The dict with the properties of an analysis instance, this is passed automatically by the getData() function.

Returns:

• instance_status - The alert level of the whole instance. This is

  important for the display of the alert level in the navigation bar on the website. -2: 'Technical issue', -1: 'info', 0: 'ok', 1: 'warning', 2: 'critical'

• time - The time the data is fetched. This can be the actual time, in

  this case now() would be the right choice, or it can be a time which is recorded in the data itself.

Return type:

(int, datetime)