Near-Real Time Reporting, Analytics and Chargeback for Compute Grids

• Are you facing exponential growth in compute grid use due to increased demands from applications like risk analysis, regulatory reporting or actuarial modeling – with a zero or declining growth budget?
• Do you need to reduce over-provisioning by exposing actual usage and utilization by grid or service to the business unit consuming those resources? Do you need the means to convince the business that their demand forecasting is resulting in inefficient use of capital investment? And correlate that with excess capacity in near real-time?
• Are you spending time building reports and queries for capacity planning, workload optimization, or long term trending? Can you accurately assess whether existing capacity is sufficient for new applications or workloads? For instance, can you determine usage drivers, performance and workload allocation by business or service by time of day?
• Can your existing tools monitor service execution and task backlogs in near-real time?
• Are you moving to a shared resource model, and need to know the cost basis for service delivery based on actual compute time and line of business usage?

Evident ClearStone provides usage, performance and financial transparency into compute grids that allows users, operational staff and architects to meet targeted performance levels at optimal resource utilization. Evident ClearStone provides the information required for improved demand management and more dynamic provisioning. Its analytics provide the objective means to determine whether resource sharing across grids is practical based on workloads, service performance, historical trends and available resource types.

Evident ClearStone is an automated reporting tool that can be customized by users. It harvests, consolidates, summarizes and analyzes data from multiple compute grids, with a catalog that addresses reporting by service, location, job and task level, for individual or multiple grids. The product is currently in production in environments up to over 80 grids and 20,000 engines, analyzing over 4M tasks/day in a single environment.

Daily Dashboard	Elapse Batch Times	Multi-Tenant Grid Requests
click on report thumbnail to enlarge

Features & Benefits

Feature/Function		Benefit
Service-Level Reporting and Analytics
Web-based reporting portal that delivers pre-defined reports to meet the needs of business users (application owners), operational staff (grid/infrastructure management), and architects (overall service design).		A single reporting solution that addresses the needs of different users of grid services as well as the operational staff.
Data Warehouse
Integrated Data Warehouse to store the enriched grid utility information and perform additional report processing through specific database Extract, Transform, and Load (ETL) functions		Data store to maintain service trending information and provide structured business reports, and an operational cache for near real-time operations reporting and SLA management. All major databases are supported
Service Accounting (Chargeback)
Rated usage for the elapsed time of task runs for each service across each grid engine. A rate class is defined as the cost to be applied to the usage of a specific engine or engine group by a service at the task level. CPU hours are rated by engine type. Three engine types and rates are defined: Dedicated Servers Scavenged Servers Harvested Desktops		Provides an economic model whereby business groups are incented to share non-dedicated grid resources, subject to being preempted by their own applications. Each engine is “owned” by a specific group, but these resources can be shared when they are free to process task from other applications. The corporate objective is to increase utilization across departmental IT infrastructure while having each department chargeback others for utilizing its available engines.