The administration of Evident ClearStone Live is performed thru the ECSL Administration Console. This is an Adobe Flex-based web application that is used to configure certain features of the ECSL Server. It can be used to configure the following system parameters:

• User Administration – provides the ability to add/update and delete ESCL system users and user credentials. This can be used to limit which users have access to the ECSL system.
• Notification Policies
  • Event Handler Plugin management - manages the deployment of custom code plugins to subscribe for events generated by ECSL.
  • Simple Network Management Protocol (SNMP) Configuration – provides the ability to configure SNMP parameters and credentials for other enterprise management systems so that alerts can be sent as SNMP traps.
  • Simple Mail Transport Protocol (SMTP) Configuration - provides the ability to configure SMTP parameters and end-user distribution lists for email systems so that email alerts can be sent to the configured users.
  • Notification Settings – defines the notification rules for which type of alerts should be sent via SNMP or SMTP alerting
• Threshold Configuration - provides the ability to set monitoring threshold policies for various elements monitored by ECSL.

Accessing the Administration Console

The ECSL Administration Console is accessible from the main web page of the ECSL product.

1. Locate the “Administration” link in the “Software” menu on the Evident ClearStone Live home page.
   
2. Or accessed directly via the Admin Console application URL: http://localhost:8080/evidentcsadmin
3. The default administrator username is "admin" with the default password as "mattison".

Note: Replace “localhost” in the URL with the fully qualified hostname of the ECSL server.

User Administration

ECSL supports role-based user administration and access control configuration. The User Administration tool allows administrators to manage user accounts and roles in the system. At minimal, each user requires a username, password, name, and email address. Access to the system features and data is managed by defining roles for both users and the data. All of the access control data is retained within ECSL's internal LDAP server.

The following screenshot illustrates the User Administration tool where ECSL Administrator can add/update user accounts and roles in the system:

Managing Users

The following steps are required to add, modify or delete users of the system:

1. To add a user, enter the required information in the “User Detail” section of the screen, i.e. user login name, password, full user name, and email address.
2. Select the role(s) for the user.
3. Click on the “Add/Update” button to save the user account information.
4. To modify a specific user’s credentials, select the user from the list and modify the appropriate values. Once all modifications have been completed, click on the “Add/Update” button to save the modified user account information to the ECSL Server.
5. Use the “Refresh” button to clear the contents of the fields and to refresh the data from the ECSL Server.
6. To test the validity of specific user’s credentials, click on the Test button.

Managing Roles

Access control to product features and data is established thru roles. The system already includes role specific to features in the product.

The following features is secured via role base access control. The relevant roles are indicated by function.

Administration and Collection Configuration Tools

• Basic access (roles(s): ECSL_ADMIN)
• All functions (role(s): ECSL_ADMIN)

Real-Time Dashboard

• Operations (role(s): user defined role in collection group*)
• Basic access (role(s): RTDASHBOARD_USER, RTDASHBOARD_ADMIN)
• Data viewing (role(s): user defined role in collection group*)
• Create a Perspective (role(s): RTDASHBOARD_USER, RTDASHBOARD_ADMIN)
• Open a Perspective (role(s): RTDASHBOARD_USER, RTDASHBOARD_ADMIN)
• Saving a Perspective (role(s):RTDASHBOARD_ADMIN)
• Removing a Perspective(role(s):RTDASHBOARD_ADMIN)

Real-Time Console for Oracle Coherence

• Basic access (role(s): RTCONSOLE_USER)
• Cluster selection (role(s): user defined role in collection group*)

Chronograph Builder

• Basic access (role(s): CGBUILDER_ADMIN)
• Create a template (role(s): CGBUILDER_ADMIN)
• Load a template (role(s): CGBUILDER_ADMIN)
• Save a template (role(s): CGBUILDER_ADMIN)
• Delete a template (role(s): CGBUILDER_ADMIN)
• Data queries (role(s): user defined role in collection group*)

Chronograph Viewer

• Basic access (role(s): CGVIEWER_USER)
• Load a template (role(s): CGVIEWER_USER)
• Data queries (role(s): user defined role in collection group*)

* - As indicated, some of the functions will limit access to data based on the role selected for a particular collection group as define in the Collection Configuration tool. The ECSL administrator may assign a specific role for a particular collection group. If assign, then all users that require access to this data in the collection group must be assigned with that role.

Policy Configuration

ECSL enables administrators to set notification policies produced by the system. For example, when threshold violations occur, ECSL will generate events for notification. These events can be delivered via email, SNMP traps, or thru a custom event handler that is provided by the customer. In order to inform users when these events occurs, the administrator will need to configure the desired notification settings and notification policies. These settings are configured using the Admin Console under the Policies configuration.

SNMP Trap Configuration

Users can define one or more SNMP trap receivers (such as an enterprise management system) for processing SNMP traps from ECSL. The SNMP traps are triggered by the ECSL server when threshold policies have been violated or real-time logs emit error log events. These triggers occur in real-time as ECSL collects and monitors the infrastructure.

ECSL only supports SNMPv2c based traps. The following screen is an example of the SNMP Trap Configuration panel.

The SNMP Configuration screen requires the following parameters to be specified for each SNMP Trap Destination:

• IP Address – the IP address of the SNMP server that will receive the SNMP trap (SNMPv2c only)
• Port – the UDP port # on which the SNMP trap will be delivered (e.g. usually, most SNMP system are configured to receive traps on port #162)
• Community String – this is the SNMP community string. This text string acts as a password and it is used to authenticate messages that are sent between the SNMP agent (in this case the ECSL system) and the SNMP manager (in this case the SNMP trap receiver within the Enterprise Management System). The community string is included in every packet that is transmitted between the SNMP manager and the SNMP agent.

In order to configure these parameters or add a new SNMP Trap destinations to an existing list:

1. Select the appropriate “IP Address” cell within an empty row by clicking on the cell within the table and enter the appropriate IP address of the destination SNMP Manager.
2. Use the Tab key to move over to the “Port” cell within the table and enter the appropriate IP Port value of the destination SNMP Manager – this is usually port #162, but can vary depending on the configuration of the SNMP Manager.
3. Use the Tab key to move over to the “Community” cell within the table and enter the appropriate Community string value required by the destination SNMP server (SNMP Manager).
4. Once all three fields have been populated with the appropriate configuration values, use the “Save” button to save the information to the ECSL Server.
5. If a second or a third SNMP Trap Destination has to be configured, repeat step #1 through Step #4.
6. To delete a specific destination after it has been saved, select the values in the table and use the delete key to remove the values, followed by the “Save” button to save the changes to the ECSL Server.
7. To delete a specific destination before it has been saved, use the “Refresh” key to get the most current stored values from the ECSL Server.
   

The following illustrates a SNMP PDU for a sample trap generated by the ECSL system:

In order to properly interpret the SNMP Traps being sent by the ECSL Server to the SNMP Manager, the SNMP Manager can use the following MIB definitions:

ECLS Enterprise MIB:

EVIDENTSOFTWARE-MIB DEFINITIONS ::= BEGIN   IMPORTS        enterprises               FROM RFC1155-SMI        Unsigned32               FROM SNMPv2-SMI;                evidentsoftware                   OBJECT IDENTIFIER ::= { enterprises 3424 } evidentsoftware-global     OBJECT IDENTIFIER ::= { evidentsoftware 1 } evidentsoftware-product OBJECT IDENTIFIER ::= { evidentsoftware 2 }   END

ECLS MIB:

EVIDENTSOFTWARE-CSL-MIB DEFINITIONS ::= BEGIN   IMPORTS        evidentsoftware-product               FROM EVIDENTSOFTWARE-MIB;                esCSL  OBJECT IDENTIFIER ::= { evidentsoftware-product 1 }   esCSLNotifications   OBJECT IDENTIFIER ::= { esCSL 1 } esCSLNotificationObjects   OBJECT IDENTIFIER ::= { esCSL 2 }         esNotifySeverityPrevious OBJECT-TYPE        SYNTAX INTEGER {               normal(0),               minor(1),               major(2),               critical(3)        }        ACCESS read-only        STATUS current        DESCRIPTION               "previous violation severity"        ::= { esCSLNotificationObjects 1 }   esNotifySeverityCurrent OBJECT-TYPE        SYNTAX INTEGER {               normal(0),               minor(1),               major(2),               critical(3)        }        ACCESS read-only        STATUS current        DESCRIPTION               "current violation severity"        ::= { esCSLNotificationObjects 2 }         esNotifyMetricName OBJECT-TYPE        SYNTAX OCTET STRING        ACCESS read-only        STATUS current        DESCRIPTION               "metric name"        ::= { esCSLNotificationObjects 3 }         esNotifyMetricValue OBJECT-TYPE        SYNTAX OCTET STRING        ACCESS read-only        STATUS current        DESCRIPTION               "the observed metric value"        ::= { esCSLNotificationObjects 4 }         esNotifyThresholdValue OBJECT-TYPE        SYNTAX OCTET STRING        ACCESS read-only        STATUS current        DESCRIPTION               "user defined threshold value"        ::= { esCSLNotificationObjects 5 }         esNotifyEntityName OBJECT-TYPE        SYNTAX OCTET STRING        ACCESS read-only        STATUS current        DESCRIPTION               "entity name which the violation occurred"        ::= { esCSLNotificationObjects 6 }   esCSLThresholdViolationNotification      NOTIFICATION-TYPE        OBJECTS {               esNotifySeverityCurrent,               esNotifyMetricName,               esNotifyMetricValue,               esNotifyThresholdValue,               esNotifyEntityName        }        STATUS current        DESCRIPTION               "A metric value has violate user defined threshold definition"        ::= { esCSLNotifications 1 }   END

Note: In order for the system to start sending SNMP Traps for threshold violations, the “Notification Settings” must be properly configured. See further below.

Email Configuration (SMTP)

In addition to or instead of SNMP Trap alerting, ECSL also supports email notification. These alerts can be triggered by threshold violations or by erroneous log events. If SMTP notification is enabled, the ECSL server will publish a SMTP email to all the configured email recipients via a standard SMTP Email Server.

To setup email notification, the system administrator must configure an SMTP host and a list of email addresses where the alert is to be sent. To control the amount of emails sent at any time, the administrator can set the frequency of the email notifications. The following is an example of the SMTP Configuration screen:

The SMTP Configuration screen requires the following parameters to be specified for the SMTP Host:

• SMTP Host – the IP address, host name or FQDN for the SMTP Server that will be responsible for distributing email alerts across the enterprise email system.
• TO: and CC: - the email addresses of the individuals to which the email alerts are to be delivered.
• From: - the email address of the ECSL Server or ECSL Administrator.
• Notification Frequency – the notification frequency is the number of minutes that the system should wait to batch consecutive notifications prior to sending an email alert. This allows multiple notifications to be sent out in a single email, thus reducing the number of emails that are generated for notification. This value defines the time duration of the “Event Period” shown in the email that is sent to the configured users (see sample below). If no alerts occur during a specific Event Period, no email is sent to the configured users.
  

In order to configure these parameters:

1. Enter the IP address, host name or Fully Qualified Domain Name (FQDN) of the SMTP Host that will relay email alerts through the enterprise email system for delivery to the intended recipient(s).
2. Use the “+” button to add user email addresses of end-users who will be receiving email alerts when alerts occur. Email alerts can be addressed to specific users using the “TO:” list and other users can be copied on the alerts using the “CC:” list.
3. Use the “-” button to remove a user from the lists, by first selecting the address of the user to be removed from the list and then clicking on the “-” button to remove the user from the corresponding list.
4. Enter the source email address of the users from which the email is to be sent in the “From:” field (e.g. in most cases this will be the ECSL Administrator).
5. Specify the notification frequency by entering the number of minutes directly into the field or by using the “Up” or “Down” arrow to increment or decrement the notification frequency. Please consider the following before setting this value:
• Negative Number – setting this parameter to a negative number, e.g. “-5” minutes, is not allowed and will be ignored.
• Zero – setting this parameter to “0” (zero) minutes will result in a single email being sent out for each alert that occurs in the system (see CAUTION below before setting this parameter to zero).
• Default Value – the default value of this parameter is “5” (five) minutes.
• Maximum Value – the maximum value for this parameter is “1440” minutes which will result in sending out a single email message that contains all the alerts for the previous 24 hrs (if any).

CAUTION: Setting the notification frequency too low or setting it to zero can result in the generation of an “email storm” during periods of large numbers of alerts. This can adversely affect the performance or operation of the SMTP Email Server/Host configured in this screen.

Batching alerts by using a larger notification frequency reduces the number of emails that are generated while still getting notified of all alerts that occurred during a specific period in a single email message. The notification frequency can be adjusted to satisfy a balance that is ideal and meets specific end-user notification requirements.

The following is an example of a threshold violation notification. It contains a summary and details of all the threshold violations that occurred during a specific period.

The email notification includes the following:

• Summary: Threshold violation events – provides a summary of the total number of CRITICAL, MAJOR, and MINOR events that occurred during the specified Event Period for the Source cluster.
• Details: Threshold violation events – provides details on each event that occurred during the Event Period (event batching) including:
  • Event severity
  • Timestamp indicating when the event occurred
  • Name of the cache where the event occurred
  • Specific Key Performance Indicator (KPI) for which the set threshold was violated
  • Set threshold value for the specific KPI and an indication of whether the threshold is a high water mark (up arrow) or a low water mark (down arrow) threshold
  • Actual KPI value at the time the event was recorded

Note: In order for the system to start sending SMTP emails for alerts, the “Notification Settings” must also be properly configured. See further below for more information.

Event Handler Plugins

About The Plugin User Interface

You will find the Plugin user interface by first launching the Evident ClearStone Live Administrative (ECSL) Console. Once you've successfully logged in to the console, drop down the Policies menu from the navigation pane and then select the Event Handler Plugins menu item. The user interface is shown below (as it appears upon first usage):

Top of Page

Connecting to the Plugin Server

Before you can perform any other activities related to Plugin management and control, you must connect to the Plugin Server. In single host solution deployments, the Plugin Server runs on the same (server) host as all the server-based services. In a more complex distributed deployment model, you'll need to know the name of the host that runs the Plugin Server. The Plugin Server runs co-resident with the Pipeline Server, so you'll need to provide that host name if you've performed a distributed host installation of ECSL.

Simply enter the hostname and press Connect. ECSL validates the Plugin Server connection. If the connection fails for any reason, a notification dialog indicating the problem will be presented.

Top of Page

Preparing a Plugin For Installation

The Sample Plugin

ECSL ships with an included pre-packaged sample Plugin that you can experiment with. This Plugin is quite simple but illustrative. The Plugin is available both in ready-to-deploy form and as a source project. The latter is distributed in a ZIP archive that can be directly imported via the Eclipse IDE (for Plugin developers). This Plugin can be characterized by the following features:

1. Single event handler (com.evidentsoft.plugin.sample.EventHandler)
2. Three (3) event channels
   • allEvents - triggered for any event including those applicable to other event channels
   • minorLowViolationClusterStorageMemoryFree - triggered when a minor low threshold monitoring cluster storage free memory has been violated
   • criticalLowViolationClusterStorageMemoryFree - triggered when a critical low threshold monitoring cluster storage free memory has been violated
3. Simple response - all 3 callbacks merely output the channel identity and received event to the standard out stream

You may freely use the sample Plugin for validation in your environment and, more importantly, as a springboard to start your own Plugin development efforts.

Top of Page

Obtaining the Sample Plugin

The plugin sample is distributed with ECSL as a self-extracting JAR archive; it is located in <ECSL-installation-directory>/add-ons/plugin-sample.jar. This archive contains a pre-built, read-to-deploy sample plugin, documentation and a ZIP archive containing an Eclipse-ready development project that contains the source code, tools, resources required to build the sample plugin. This development project provides a springboard for you own custom plugin development work.

To run the self-extracting archive, you will need to have made previous arrangements so that Java 6 JRE is accessible (i.e. the bin directory is in your PATH and JAVA_HOME has been established in your environment). The exact details of the extract vary slightly by platform but generally you will want to be running the platform's supported command environment (e.g. a CMD window on MS Windows platforms, BSH on Linux etc.). To extract the sample just run the following command:

java -jar <ECSL-installation-directory>/add-ons/plugin-sample.jar

The extraction process should indicate the target directory where the extraction takes place, typically this is the directory called ecsl-plugin-sample/ which is created as a subdirectory of the current user's HOME directory (e.g. on Windows this might be c:\Documents and Settings\don\ecsl-plugin-sample). The extracted sample should contain documentation (docs/), a deployable sample Plugin (/lib) and an Eclipse-ready project (plugin-project.zip). If your platform and JRE supports the desktop facility, then the extractor will launch a browser to display the content of the Evident ClearStone Plugin Developer’s Guide.

Top of Page

Plugin Resources Location

Plugins are generally installed from the Plugin Auto-Deployment Directory or a sub-directory. This provides containment for all deployed plugins, insures that plugins are started when ECSL starts and facilitates the deployment interface provided by the console. Generally speaking it is a good idea to create an immediate subdirectory for each plugin though this is not strictly required. It does, however, avoid collision and confusion among the set of dependent jars that plugins may employ. The auto-deployment directory is typically found here:

<ECSL-root>/config/plugins

If the Plugin Auto-Deployment Directory does not exist, you may need to create it as specified above. Create a sub-directory of the Plugin Auto-Deployment Directory called sample/. Copy the ecsl-event-plugin.jar and the log4j-1.2.15.jar to the sample/ subdirectory. From the console UI, choose the Install New Plugin tab. The set of available plugins should be shown. The sample plugin jar (ecsl-event-plugin.jar) should be present. If not, insure that the log4j-1.2.15.jar is also present in the same directory as it is required for the scan to recognize the ecsl-event-plugin.jar is a viable candidate plugin. If all proceeds accordingly, the screen should now look like this:

Notice that the lugin jar (ecsl-event-plugin.jar) appears in the list of candidate plugin jars. (We refer to a Plugin jar as a candidate until it is successfully installed.)

Top of Page

Installing a Plugin

Before we install the (sample) Plugin, recall that we'd located the jar(s) that the Plugin requires in a special auto-deployment directory. Besides becoming an available candidate that is visible (in the UI) for installation, ECSL regards any Plugins that have been located here as candidates for un-attended start. This means that ECSL will automatically install and start all plugins that are found in the auto deployment directory or in any sub-directory. This auto-deployment step occurs only at ECSL startup. Since we've just located the sample Plugin here, it won't automatically install until we explicitly request that it do so (or we restart ECSL).

We'll prepare to install the sample Plugin by first selecting it from the list of available jars and then by giving it a Plugin ID of sample. Once that's done we merely need to press the Install button to begin the Plugin installation process. If installation proceeds successfully, the Plugin will be listed in the Plugins table. The entry identifies the Plugin by its ID, indicates the (primary) Plugin jar and the Plugin state. Plugins that install properly undergo several state transitions. After installation the Plugin remains in the loaded state until the Plugin framework prepares the internal "machinery" required for event dispatch (delivery). Once this has been established the Plugin transitions to the running state. Shown below is an example of what the user interface might show. Note: we've selected the Plugin from the Plugins list and have selected the Plugin Details tab to observe the information available. We'll say more about this in a moment.

Note: Plugins are given a Plugin ID that must be unique among all known Plugins. This user-provided moniker is merely a distinguishing reference to the instance of the Plugin and is otherwise not important to the Plugin framework. Since there is no restriction on installing the same Plugin more than once, its distinct identity allows us to readily differentiate all Plugin instances including those representing the same underlying Plugin (code).

Top of Page

Viewing the Plugin Details

Plugin details are shown by first selecting the Plugin you wish to work with from the plugins table, then by selecting the Plugin Details tab. To update the details statistics while looking at this tab, merely press the Refresh Stats button as desired. The figure below shows the state of our Plugin after it has had an opportunity to run a bit and to process some events.

Information that is common to all plugins is shown on the left hand side of the details view. This includes the plugin's ID, a count of the events delivered, the time of receipt of the most recent event and the average time spent in the plugin code that is specific to the handling of each event. (The average handler time benchmark is a measure of the plugin's efficiency in handling events.) A fraction shows the greatest number of events that were simultaneously waiting to be processed (2 in our example) and the greatest number that can be waiting (64) before events are lost. The event lost count, if any, is also presented.

A Plugin may be designed and implemented such that it handles more than one distinct class of event. In Plugin parlance this is known as a channel. Each event channel is associated with a name and a filter expression that isolates the events of interest to the channel. The Plugin implementation provides one (or more) callbacks for each channel so that custom Plugin code may be executed in response to a channel event. On the right hand side of our details view, the table shown provides information about each channel and the distribution of channel-matching events that may have been received. For example, the minorLowViolationClusterStorageMemoryFree channel has triggered the Plugin callback for this channel a total of 4 times.

Note: It's important to note that channel definitions may overlap. This is illustrated here by the allEvents channel which is active for every event. The Events Delivered count is equal to the total events delivered to Plugin callbacks across all channels but does not necessarily represent the number of unique events received by the Plugin. In our sample Plugin , the allEvents channel identifies the number of distinct unique events, seven (7) of these were violations and 11 were others. Your plugin may not include an "all events" channel, so be careful about drawing conclusions from these statistics.

If you want to purge the collected statistics and restart from scratch, just select the Plugin of interest from the Plugins table and press the Reset Stats button. This will "zero" all statistics.

Top of Page

Pausing and Resuming

Plugin operations, i.e. receipt and handling of events for any defined channel, may be temporarily interrupted as desired. To do so, select the Plugin from the Plugins table and, if its state is currently running, press the Pause button. The Plugin state should show stopped and the Pause button is replaced with a Restart button that can be used to resume event handling in the selected (stopped) Plugin. See below:

Put images/ui-stopped-plugin.png

Notice that, while stopped, the Plugin buffers a limited number of events. Here, we've buffered up to 8 events that are pending delivery. A restarted Plugin will receive all pending events immediately after resumption. In the event that the dispatch queue reaches its limit while a Plugin is paused, some events may be lost. These are indicated by the events lost count. The queue keeps the newest events when it reaches full capacity. The oldest occurring events are lost.

Top of Page

Un-Installing a Plugin

A Plugin may be un-installed at any point in its lifecycle merely by selecting it from the Plugins list and then by pressing the Uninstall button. After being uninstalled the Plugin is no longer shown in the Plugins list and all details are lost.

Note: ECSL makes no guarantee that the Plugin will not be interrupted during the handling of an event or that any pending events will be processed. It's important to note that had the Plugin been installed from the auto deployment directory as described, the Plugin will still be restarted upon ECSL restart. To avoid this condition, you'll need to remove the Plugin jar (and any additional jar resources) from the auto deployment directory (or subdirectory). Following un-installation, the jar resources may be briefly held by ECSL (until garbage collection). If you have trouble removing these jars, simply wait a bit and try again.

Top of Page

Autonomous Behavior

ECSL supports autonomous Plugin installation and startup for any Plugins located in the auto deployment directory (or a subdirectory) as we've previously described. Furthermore, ECSL maintains a Plugins Registry that is part of its persistent configuration that identifies all Plugins that are known to have been installed either explicitly (using the user interface) or via the auto deployment mechanism. Plugins that are incorporated to this registry are candidates for autonomous behavior even if their resources are not located in the auto deployment directory.

If you wish to remove all Plugins from consideration, you need to (a) remove any Plugins from the auto deployment directory (or subdirectory) and (b) remove the Plugins registry. The registry is stored in an XML configuration file that is typically found here:

<ECSL-installation- directory>/config/perco/perco$description=Clearstone%20Plugin%20Extensions%20Registry,instance=global,name=plugins,scope=global

(Note: %20 represents the SPACE character.)

Top of Page

Managing Plugins

Plugins may be explicitly managed (i.e. installed, un-installed, started, stopped, monitored) using the console as we've previously shown. You must make arrangements to deliver the Plugin jar(s) to the target host system (ECSL Plugin Server). Future versions of ECSL will likely support a Plugin upload capability.

Auto Deployment

We've already briefly discussed the Plugin auto deployment capability. The Plugin auto-deployment directory is controlled by a property of the Lodestar configuration (found in <ECSL-installation-directory>/tomcat/webapps/lodestar/conf/lodestar.properties). The relevant configuration options are shown below:

# PLUGIN-CONTROL
plugins.enabled = true
plugins.autodeploy.dir = ${evident.root}/config/plugins

The default directory, controlled by plugins.autodeploy.dir, is shown. In the event that you change this directory, an ECSL restart is required. Furthermore, you must keep in mind that the deployment registry maintained by ECSL may reference Plugins that were previously deployed to an old auto-deployment directory (or sub-directory). To insure that this is not the case, you should remove the Plugins registry (found here:

<ECSL-installation-directory>/config/perco/perco$description=Clearstone%20Plugin%20Extensions%20Registry,instance=global,name=plugins,scope=global)

The Plugins registry, furthermore, may reference Plugins that are outside the scope of the auto-deployment directory. In general it is recommended that you first copy Plugins (and their dependencies) to the auto-deployment directory (or sub-directory) before deployment. Doing so avoids referencing Plugins that may be orphaned if the Plugins registry is deleted.

Enabling / Disabling

The plugins.enabled parameter represents a master control or "kill switch" for Plugin control. Setting this to false (and restarting ECSL if running) defeats the deployment of all Plugins but does not otherwise affect the auto-deployment directory contents or the contents of the Plugins registry.

References and Other Resources

For a great deal more information about Plugins and, specifically, Plugin development, please see: Evident ClearStone Live - Plugin Developer's Guide

Notification Settings

The ECSL notification feature allows administrators to control how users are notified of events. Events can be sent to users as SNMP traps, emails via SMTP, or written as entries in log files on the ECSL server. The administrator specifies whether or not each type of notification (i.e. SNMP trap, SMTP email, LOG entry) will be sent based on the severity of the event (i.e. NORMAL, MINOR, MAJOR, CRITICAL). Enabling or disabling notifications by severity and notification type is used to control the volume of event notifications generated by the system. Each notification type can be configured separately thru its own set of severities that can be enabled or disabled as illustrated in the following figure:

To configure notification settings:

1. Enable or disable all notifications by selecting the “Notifications Enabled” check box in the upper left-hand corner of the table. This is a "master switch" for enabling or disabling notifications.
2. Select or clear the check boxes in the table to enable or disable event notifications by severity and notification type (i.e. SNMP, SMTP, and LOG).
3. Click on the “Save” button to save the settings to the ECSL Server.

If Log notification is enabled for one or more log levels, the ECSL server will produce a consolidated log file on the server consisting of all log events for the enabled log levels. This file will roll over every midnight. Log events from each grid will be consolidated together in the same log file.

Note: The Notification Enabled setting must be selected before any notification can occur. If this setting is enabled and there is no valid configuration information for SNMP or SMTP, no alerting will occur. Valid SNMP or SMTP configuration is required notifications to work properly (see previous section).

Threshold Policies

The Threshold policy feature enables users to define performance thresholds for any metric that is managed by ECSL. This feature is accessible thru the Administration Console under Configuration.

New Threshold Policy

To create a new policy, click on the "New" button at the bottom of the Threshold Policy screen. A threshold policy consists of the following configurable elements:

• Data Type - the specific data model for this policy base on the supported information models.
• Policy Name - a customizable name for this policy.
• Policy Description - a customizable textual description for the policy.
• Resolution - a customizable set of troubleshooting steps to resolve a threshold breach. This is for information only and will be display in the fault management panel when a threshold event is viewed.
• Enable - set the threshold to be immediately active or disabled after creation.
• Event Description - a customizable message for notification. The text is a parameterize string that can contain variables that will be replace when an event is produced when a threshold violation occurs. The valid parameters are:
  • ${source} - identifies the source of the measured element that is causing the threshold breach (i.e. process id, hostname, method call)
  • ${kpi} - the metrics configured for this policy
  • ${threshold} - the configured threshold that was breached
  • ${value} - the measured value that incurred the breach
• Metric - the selected metric from the specified data type.
• Thresholds - numeric threshold boundaries (low and high watermarks). The three boundaries are MINOR, MAJOR, & CRITICAL.
• Filters - apply a filter on one or more non-numeric fields. The fields are directly associated with the selected data type. For any selected field, you can apply a filter that either matches, starts with, contains, or ends with text specified in the value.

Once a policy is created or updated, click on the "Save" button. As policies are created, they are added to the list of Threshold Policies tree. The polices in the tree are automatically grouped by Data Type and Metric. Therefore is easier to identify what policies have been set for different metrics.

Enabling/Disabling Policies

To enable/disable multiple existing threshold policies, check or uncheck the items in the tree and click on the Enable Policies button at the bottom of the tree.

Removing Policies

A policy can also be permanently removed by selecting on the policy in the tree and clicking on the "Remove" button at the bottom of the screen. The removal only applies to one policy at a time.

As thresholds are created or updated, they take effect immediately (if enabled). As thresholds are breached, the violating KPIs in the corresponding Real-Time Dashboard perspectives will be displayed in different colors (depending on the severity level breached). Meanwhile, all threshold violation events also appear in the Fault Management perspective in real-time. If configured, ECSL can generate events that would result in email alerts, SNMP traps, and callbacks into a plugin code.