The Evident ClearStone Information Model Builder is a configuration tool that is used by administrators and developers to manufacture and manage the Information Models for ECSL. An information model consist a data dictionary that defines the structure of the data for collection, processing, storage, and presentment. The ECSL platform consists of Collectors, Pipeline Engines, Data Services, and Real-Time Dashboard applications that depends on these configurations to apply the application logic during data collection and transformation.

Developers and administrators can construct information models to monitor third party or custom JMX-enabled applications. This enables ECSL to monitor new applications with little or no additional coding effort. Depending on the application, the mbeans can expose application statistics or business level KPIs that can be monitored and tracked by ECSL. Users can leverage ECSL's built-in features for:

• Caching real-time metrics for 24 hours
• Threshold based monitoring and alerting
• Real-time customizable visualizations
• Custom hooks for event handling
• Historical data export and warehousing
• Integrate performance monitoring across multiple disparate applications

The Information Builder currently supports the creation of new information models based on JMX data sources. ECSL supports collection from standard and open mbeans types. Currently, support for dynamic or model mbeans is not available. Although creating new information models from non-JMX data source is possible, that would require additional collaboration with Evident Software's professional services.

Note: This tool should be used by trained and knowledgeable users who are deeply familiar with the managed environment and with the constructs of an ECSL Information Model as documented below.

About Information Models

As described above, an information model consist a data dictionary that defines the structure of the data for collection, processing, storage, and presentment. Information Models are organized by data dictionaries of one or more heterogeneous type definitions. There can be multiple data dictionaries of information models configured with ECSL. A single data dictionary can consist of type definitions that are designated as:

1. Collection type - used by the Collectors for querying data from a target. For example, if the target was a JMX mbean, this would contain the object name patterns and attribute names for each type definition. Once data is collected, the remainder of the system is agnostic to the collection data source.
2. Extension type - used by the Pipeline Engine to apply additional data transformation on the collection type. Transformations typically include applying mathematic expressions on the metrics (i.e. ratio calculations, rates, scaling, etc.). For more supplicated data transformation and/or correlation, users may need to program a Pipeline plugin that is called upon to process the collection type data. The Pipeline Engine produces a new set of records that may contain more or less attributes than the collection type.
3. Aggregation type - used by the Pipeline Engine to apply aggregation logic to collection types and/or derived types. Users need to select which attributes to group by and the method of aggregation for the metrics for the aggregated records. The Pipeline Engine produces a new set of aggregated records.
4. Clearbase type - used by the Clearbase Publisher to publish record to the ECSL data warehouse (Clearbase). The Publisher consolidates and aggregates records from the ECSL OpCache and writes them to the database. Users can pick and choose which attributes and metrics that would be practical to store in the data warehouse.

For each type definition defined, the data can be cached, exported, and/or published to the ECSL data warehouse. Those type definitions that are marked as chartable will be cached and displayable. Those that are Clearbase type are destined for the data warehouse. It's possible that some types are never cached since they may be used for the basis of aggregation or correlation.

Each data dictionary is stored as an XML file (“<INSTALL_DIR>/config/perco/perco$instance=<name>,name=datadictionary,scope=global.xml”). This is shared by all the ECSL applications on each server. For a multi-server environment, it's important to distribute any information model changes across the servers.

The Information Model Builder is a separate web application that can be deployed on any ECSL server. Evident recommends deploying this on a separate development/test environment that has access to the target managed environment. Updates to Information Models should be tested and validated in this test environment before deploying onto a production ECSL system.

The Information Model Builder can be downloaded from the same site where the ECSL product was downloaded from. The web application is called imbuilder.war. The web application archive should be deployed into <INSTALL_DIR>/tomcat/webapps directory of an existing ECSL system.

Access to the Information Model Builder requires a web browser with an Adobe Flash Player plugin which can be downloaded from Adobe's website. The application URL is http://localhost:8080/imbuilder.

Note: Replace “localhost” in the URL with the fully qualified hostname of the ECSL server where "Data Services" option was installed.

Construct a New Information Model

Create new data dictionary for Apache Tomcat 6

The following exercise demonstrates how to create a new information model for monitoring session activity for web applications deployed into an Apache Tomcat 6 process.

1. Upon launching the Information Model Builder, the following Data Dictionary selection screen is presented.
   
   
   
   
2. In this example, there are two pre-existing data dictionaries (Java and Oracle Coherence).
3. To create a new data dictionary, click on the "+" button.
4. Enter the name of the new data dictionary: "Tomcat6"
5. Enter a description of the new data dictionary: "Data model to support Apache Tomcat 6"
6. Data dictionaries can contain user-defined static attribute known as "tags". These tags are inherited by all the type definitions defined in this data dictionary. Tag values are specified as collection configurations are defined. In this example, let's define a new tag to denote the operating environment (i.e. production, dev, uat, dr, etc.) for the Tomcat instances. Since this type of information is generally not available in the standard Tomcat or Java mbeans, using a tag is practical here. Enter the following information for this new tag:
• Name: Environment
• Description: Operating environment for this Tomcat instance (PROD, DEV, UAT).
• Label: Environment (This is the display label for the user interface)
• Filterable:
• Groupable:
7. Save the new data dictionary by clicking on the Save button.
   
   

Create new collection type

1. After the new data dictionary is created, let's add the first collection type.
2. If you are on the Data Dictionary selection screen, select the new Tomcat6 data dictionary.
3. From the Type Definitions Panel, click on the "New Type Definition" button or the "+" button to create a new collection type.
4. The user interface displays two panels: Mbean Browser and Type Details.
   
   
5. The Mbean Browser panel is used to connect to an active mbean server to browse for mbeans in the target environment. To connect to a remote mbean server, specify the RMI host and port number (host:port). Include the JMX user credentials if the mbean server is secured. In this example, specify the remote Apache Tomcat server to connect to.
6. Once connected, the Mbean Browser would expose the hierarchy of the registered mbeans. Use the Mbean Browser panel to traverse navigate thru the mbean hierarchy.
7. In this example, we'll setup ECSL to monitor the session activity for each web application hosted on the Tomcat instances. Therefore, locate the Session Manager mbeans. This is locate under Catalina:type=Manager.
   
   
8. In this example, there are 14 Session Manager mbeans registered (one for each web application). Pick any one of the Session Manager mbeans for any web application to view the Session Manager's attributes.
9. For session activity, select the following attributes: activeSessions, expiredSessions, maxActive, maxActiveSessions, rejectedSessions, sessionAverageAliveTime, sessionCounter, & sessionMaxAliveTime. Unselect the other ones that are not required for this collection type.
10. Once the attribute are checked or unchecked, then click on the "Add To Type" button. This automatically populates the new collection type with the following:
• Key attributes - based on the tags define in the current data dictionary and properties in the object name of the mbean.
• Non-key Attributes - attributes (numeric or non-numeric) that are not used for charting or thresholding
• Metrics - numeric attributes that are used for charting and thresholding in the user interface
11. In this example, 4 new key attributes are created by default. The keys are automatically chosen based on the tags define in the current data dictionary and the object name properties for the selected mbean.
    
    
12. In this case, there are no additional attribute that are not considered as a key or metric.
13. In this example, 8 new metrics were selected and added to the new collection type. The Information Model Builder automatically includes the mbean's attribute description and type.
    
    
14. By default the sessionCounter is sessionCounter is a cumulative counter. This counter would continuous increment until it's been reset or until the web application has been restarted. Therefore, it would be beneficial to apply a delta calculation for this metric.
15. Now that the key and metric attributes have been selected and updated. The next step is provide a name for this collection type ("Web SessionStats") and description ("Web application session activity").
16. To enable this new collection type ("WebSessionStats") to be visible in the Real-Time Dashboard, the Chartable option should be checked. If this is not checked, then this data will not be visible in the user interface. Sometimes a collection type is only needed for correlation and aggregation. Once the Chartable option is checked, the ID Label is updated with a predefined string pattern. This pattern is a formatting pattern is used for uniquely identifying each record within the dataset.
17. Now that this collection type is chartable, additional properties will appear for the attributes in each of the tables. The key attributes includes the following new properties:
    
• Label - user friendly display name for the UI
• Filterable - allows the attribute to become available for filtering data in the UI
• Groupable - allows a key to be used as an aggregation key

  

18. The ordering of the attributes can changed by dragging one attribute below or above another attribute.
19. The metric attributes includes the following new properties:
• Label - user friendly display name for the UI
• Formatter - certain metrics can be formatted (byte counters, time measurements, percentages)
• Units- units of measurement for the metric
  
  
20. Instead of collecting data for a single Session Manager mbean (Catalina:type=Manager,path=/,host=localhost), it would be preferable to collect from all the Session Manager mbeans for each instance. This requires applying an object name pattern (Catalina:type=Manager,path=*,host=*). This pattern is used by the ECSL JMX Collector to query all mbeans with this pattern in the remote mbean server(s).
21. The ordering of the attributes can changed by dragging one attribute below or above another attribute.
22. Once these steps are completed, the new collection type can be saved by clicking on the Save button.
23. Clicking on the "Select Data Dictionary" button displays the Data Dictionary selection screen. You'll notice the Tomcat6 data dictionary is now listed.

Create new aggregation type

The next exercise demonstrates how to aggregate the web session activity across multiple Tomcat instances for the deployed web applications.

1. If you are on the Data Dictionary selection screen, select or double-click the Tomcat6 data dictionary. The following screen is displayed:
   
   
2. Double-click on the WebSessionStats type definition.
3. Click on the Derived Type(s) drop-down menu button and select New Aggregation Type.
   
4. The screen would be split into two upper and lower panels. The upper panel is the type definition for the current WebSessionStats and the lower panel is used to construct the new aggregated type.
5. In the lower panel, the builder prefills the new aggregation type with default a name, description, ID label, and attributes. Now it's time to update these settings for this new aggregation type.
   
   
6. You can change the name and description of the new type definition.
7. Since we intend to aggregate the session metrics across all instances, the records should be grouped by Environment and Path. Therefore, uncheck Host and Type in the Keys tab.
8. Update the ID Label from ${Environment}_${type}_${path}_${host} to ${Environment}_${path}. This will be used to uniquely identify each aggregated WebSessionStatsAgg record by Environment and webapp name.
9. In the Metrics tab, the default aggregation method is "Average". Let's walk thru each metric and decide if we want to aggregate it and if so how and what it should be named.
• sessionMaxAliveTimeAvg: Let's remove this from the aggregation. Select this metric and click on the "-" button.
• activeSessionsAvg: Rather than taking an average, let's total up the activeSession across all WebSessionStats records group by Environment and Path. Change the calculation from Average to Sum. Change the name to activeSessionsTotal and label to Active Sessions.
• sessionCounterAvg: Let's total up all the new sessions across all WebSessionStats records group by Environment and Path. Change the calculation from Average to Sum. Change the name to newSessionsTotal and label to New Sessions.
• sessionAverageAliveTimeAvg: Let's keep this. Since this is a measurement of time, update the Formatter with timeinterval and Units with ms.
• maxActiveAvg: Let's remove this. Click on the "-" button.
• maxActiveSessionsAvg: Let's take the maximum value across all WebSessionStats records group by Environment and Path. Change the calculation from Average to Max. Change the name to maxActiveSessionsPerHost and label to Max Active Sessions (Host).
• expiredSessionsAvg: Let's remove this. Click on the "-" button.
• rejectedSessionsAvg: Let's total up the rejected sessions across all WebSessionStats records group by Environment and Path. Change the calculation from Average to Sum. Change the name to rejectedSessionsTotal and label to Total Rejected Sessions.
10. In this example, there is no need to change the formatter fields. However, if you are working with other metric types such as byte counters and percentages, the following formatters and units can be applied.
    
    

    Formatter	Units
    bytes
    mbytes	MB
    timeinterval	ms
    percent

11. Once the updates are finished, click on the Save button in the lower panel to save the new aggregation type definition.

Updating type definitions

Once the type definitions are saved to the data dictionary, they can be updated one by one with the Information Model Builder.

1. To navigate to the Data Dictionary selection screen, click on the "Select Data Dictionary" button at the top of the screen.
2. From the Data Dictionary selection screen, select or double-click on the target data dictionary.
3. From the list of configured type definitions, double click on any row to begin updating the type definition.

Validating and Deploying Information Models

Validating type definitions

When information models are created or updated, they need to validated in a test ECSL environment. The recommended procedure is as follows:

1. Before making changes to an information model, please backup the <INSTALL_DIR>/config/perco directory on the ECSL server.
2. Information models are only loaded upon restarting ECSL. Therefore, it's likely there will be a number of restarts to ensure the type definitions are producing the right information.
3. Start testing with one JMX data source to ensure data can be collected, processed, and presented for the modified information model.
4. The JMX Collector(s) will need to be configured with the JMX data sources and collection types defined in the collection configuration. Please consult here for more information.
5. Monitor the ECSL logs for any problems with collecting, processing, and presenting the data.
6. Use the ECSL Real-Time Dashboard to validate the new chartable type definitions are exposed. If there is data available, the Real-Time Dashboard will display records in the data grid.
7. Once the data validation is confirmed and no errors are emitted by ECSL, then deploy the configurations to the production ECSL servers.

Publishing type definitions

When information models are ready for deployment into a production ECSL environment. The recommended procedure is as follows:

1. Stop the ECSL processes on the production system(s) (including any remote Collectors)
2. Backup the <INSTALL_DIR>/config/perco directory on each ECSL server.
3. Upload the new/modified data dictionary files into the <INSTALL_DIR>/config/perco directories across each ECSL server.
4. Configure the designated remote Collector with the new collection configuration for the new collection types.
5. Restart the ECSL processes:
   • Starting with the server that is running ECSL OpCache and Pipeline Engines
   • Followed by the Collectors