Articles tagged nosql db

The views expressed in this blog are strictly personal, and do not necessarily represent the views of Evident Software.

By Bill Nigh

Check it out.

Our Evident ClearStone 5 Press Release.

By Bill Nigh

NoSQL DB logging is characterized mainly by point solutions, reflecting the varied points of origin of relatively new technologies. The way is clear for a monitoring platform that can integrate logging and monitoring of  Distributed Cache and NoSQL databases, and wouldn’t it be nice to monitor practically any resource anywhere in the enterprise?

Evident ClearStone 5.0, with DevOps friendly pricing, gives you the ability to monitor resources through channels such as Linux SAR, PHP, Perl, Ruby and other scripted feeds using ODI (our new REST API), joining our existing JMX-based approach for monitoring JVMs.  The performance metrics are captured and formed in Neo4J as graph objects, then stored in Cassandra as time series data. This rich storage model opens up the possibility of more and more interesting visualizations of events relationships and entities.

The new release of Evident ClearStone continues our efforts toward a more integrated approach to monitoring application performance throughout the entire enterprise application stack and operating environment.

Learn more about our performance monitoring solution for Java, NoSQL and web servers.

By Bill Nigh

Why would an organization run both Memcached and Cassandra, and how could one monitor both Memcached stats and Cassandra performance in a convenient way?

There are a number of reasons for running both technologies. Maybe a recent merger has brought both technologies in house. Maybe Cassandra is being used for time series data, such as we do with Evident ClearStone, our NoSQL DB monitoring tool, and Memcached is the front serving cache. Cassandra is pretty fast as it is, so the combination is unlikely, but if such a combination were found, then the monitoring solutions would typically be point solutions. Adding the separate consoles for these products to monitoring applications for JVM such as JConsole, WebLogic, JBoss and others, and the common problem of the NoSQL space hits you in the face: a lack of integration within a single dashboard.

Evident ClearStone’s Real Time Dashboard can display JMX-based, Linux SAR, php, perl, ruby and other scripted feeds on the same page. With this functionality, you can at least begin to glimpse the outlines of a more integrated approach to monitoring application performance.

Learn more about our performance monitoring solution for Java, NoSQL and web servers.

By Bill Nigh

In a recent post, I shared the story of how Evident ClearStone had been instrumental in helping an enterprise see a pattern of interaction between an organization’s application tier (WebLogic) and data caching tier (Coherence) that was bringing a website to its knees during periods of high demand caused by promotional campaigns. Here are some tidbits on WebLogic Server performance tuning from some of the troubleshooting cycles of that engagement.

It turns out that the organization was co-locating WebLogic Server and Coherence nodes on the same server. When there were performance problems it was important to understand which of the tiers was causing the problem.

We did not see on the server (a heavy-duty Solaris box) any telltale CPU usage spike. CPU utilization was around 10%. However, when we looked at the individual cores, one core repeatedly was maxing out, and this, remember, on a multi-core server. We used prstat (process stat) to get proper granularity of information on the relationship of process to CPU to core utilization. Working from this discovery, we could then isolate for the Coherence tier.

But what was taxing the Coherence tier during those periods of peak demand in response to that promotion? It turns out that everyone who got the email blasts was being directed to the same page on the website, rather than partitioning based on information that could be gleaned from website customer profiles. Because everyone was going for the same page, ‘hot keys’ were being created. In such scenarios, the ‘hot key’ refers to a popular piece of information; that information only exists in one node (unless one counts the backup node, which is not visible to read requests). Because the same node must satisfy all requests, it can get overwhelmed and fail. Staff analyzed the URLs that were directing people to the site, and were able to trace the requests back to the email blasts tied to the promotion.

So the root cause analysis started at the server level, then focus shifted to the application, and within the application to the actual objects being retrieved, but the problem was caused by the promotional email campaign. The problem, in other words, began in a functional area of the organization that was oblivious to the infrastructure implications of their actions.

With NoSQL DB, unlike SQL, you don’t start with a normalized schema; rather, the data is (should be) structured in response to what you want. Likely queries are part of use cases, and you’re concerned with optimizing for reads or writes (or maybe a balance of both). In the case we’ve cited, the integrity of the system was compromised because of lack of understanding of how the email blasts were creating ‘hot spots’. And, since an RDBMS with all its mature and tested query optimization was not used, a possibly unexpected burden now existed.

Learn more about our performance monitoring solution for Java, NoSQL and web servers

By Scott Barnett

We are about to release our newest version of ClearStone.

We have learned a lot over the past few years, serving customers in the Oracle Coherence and Data Synapse markets, as well as over the past 6 months as we turned our focus to NoSQL DBs – see our product section on NoSQL DB Logging and Reporting.  ClearStone 5.0 introduces the concept of Application Performance Management for NoSQL and MORE.  What do we mean by NoSQL and MORE?

First, we firmly believe that NoSQL (or whatever it is ultimately called) will be the next major tier of the application stack. As such, it will need all of the tools, from development and deployment to, yes, management. There are some basic monitoring capabilities available today for NoSQL DBs, but they are hardly enough.  They lack the detail, scope and actionable capabilities that most DevOps folks are used to.  At the same time, monitoring NoSQL by itself is not satisfactory – in order to get a holistic view of any web, cloud or enterprise application, developers and operations will need to see what’s going on at all levels of the application – from the system level to in-memory cache and everything in between.

So, ClearStone 5.0 was developed to move us even more firmly in the direction of holistic application management with a direct focus on NoSQL. It’s a release intended to address the reality that many applications using NoSQL are leveraging cloud assets and need a tool that can handle dynamic environments combining traditional Java stacks, NoSQL data caching, and virtual cloud environments.  Under the hood, we’ve replaced our internal caching mechanism in 4.x with a combination of Cassandra and a graph database that allow us to store more information, and provide better correlation/relationship mapping of assets between tiers in the application.   We kept (and continue to improve) our really slick user interface. We’re working to provide additional capabilities for people to get custom reports and specific views into their data.

The biggest change in 5.0 is the introduction of an API (we’re calling it the RESTful Data Interface) that lets you instrument metrics, KPIs, and SLAs for any IT asset using a scripting-friendly, XML-over-HTTP interface. We now support built-in collection of application-level metrics from platforms such as JBoss, WebLogic, and Tomcat and will introduce built-in collection of system-level metrics, such as UNIX SAR (system activity recorder). The new back end enables correlation of all collected metrics and events, including custom metrics/events, and real-time visibility into historical performance data without requiring a relational database.

We’ve been embracing the increasingly popular DevOps model, so that both developers and operations can take advantage of the platform. Specific new features for developers (especially during testing) include the ability to:

  • Define SLAs and conditions for alerting
  • Analyze performance, capacity, and utilization across multiple applications/systems
  • Tag resources into logical groups for easy reporting
  • View both real-time and historical performance data from a single interface

ClearStone 5.0 dramatically extends what we can manage, and provides much more flexibility and openness via the API. The architecture changes also allow us to become “cloud-friendly” and has prompted a dramatic pricing change where we will charge via a subscription based on the number of “resources” you are managing, not by server/core counts. By the way, the first 10 resources will be free! Beyond that, pricing will start at $10/resource/month, so you can start small and work your way up.  Now, not only the product is elastic, our pricing is too.  For those larger companies that only want to give us large sums of money, we still support ELA and all-you-can-eat prices for environments greater than 1,000 managed resources.

We will be beta testing 5.0 starting February 14, and plan to GA in mid-March. To sign up for the beta, go to http://www.evidentsoftware.com/clearstone-5-0-beta-program and fill out the form.  We’ll be taking the first 50 users in the beta program, and I hope you’ll be one of them!  I look forward to your feedback on our new software and our new direction!

Related Articles:

Enhanced by Zemanta

By Bill Nigh

What does a traditional RDBMS programmer or architect need to understand to be productive with NoSQL (Not-only SQL technologies) and DCP (data caching platforms)?

I asked this question of our development team.

Here’s their list of things to know:

  1. Understand how ACID compares with BASE (Basically Available, Soft-state, Eventually Consistent)
  2. Understand persistence vs non-persistence, i.e., some NoSQL technologies are entirely in-memory data stores
  3. Recognize there are entirely different data models from traditional normalized tabular formats: Columnar (Cassandra) vs key/value (Memcached) vs document-oriented (CouchDB)  vs graph oriented (Neo4j)
  4. Be ready to deal with no standard interface like JDBC/ODBC or standarized query language like SQL; every NoSQL tool has a different interface
  5. Architects: rewire your brain to the fact that web-scale/large-scale NoSQL systems are distributed across dozens to hundreds of servers and networks as opposed to a shared database system
  6. Get used to the possibly uncomfortable realization that you won’t know where data lives (most of the time)
  7. Get used to the fact that data may not always be consistent; ‘eventually consistent’ is one of the key elements of the BASE model (I see this latency issue all the time in Twitter, in ‘Followers’ list)
  8. Get used to the fact that data may not always be available
  9. Understand that some solutions are partition-tolerant and some are not

These attributes vary from one system to another. It’s as important to understand the differences among NoSQL technologies as it is important to understand how they differ from a traditional RDBMS.

Here is a pretty good list of the many NoSQL products, from a respected member of the community, Alex Popescu.

Learn more about our performance monitoring solution for Java, NoSQL and web servers.

By Bill Nigh

NoSQL DB logging and reporting are challenges, for reasons discussed in this post. I recently spoke with Evident Software veteran Don Jeffery (@drjmun on Twitter) about those challenges and how Evident ClearStone (ECS) addresses NoSQL DB logging and reporting.

Metrics Collection

ECS collection (using JMX and ODI, our RESTful API over HTTP) creates Neo4j graph nodes from harvested and derived data in the form of resource metrics, resources, relationships and events. A unique identifier is generated for each Neo4j graph node, regardless of type. This identifier can be used to retrieve information from Apache Cassandra 0.7, which we employ as a time-series database, thus supporting current and historical performance monitoring visualizations within customizable perspectives. This product design allows ECS to chart performance metrics and display events such as threshold violations. (A nice writeup of our use of Neo4j and Cassandra was done in a blog post by our CTO, Ivan Ho, and got good play on DZone).

In ClearStone, a Neo4j node represents an instance of any entity we choose to track. The Neo4j node may have attributes that relate to, say, a host, such as an IP address, or data that somehow through heuristics lets us calculate how many processors may be on a piece of hardware. Other resources, possibly from other technologies, may also have information that helps us confirm that new host entity, create an instance of it, and populate it opportunistically as more information comes in. The incredibly free form nature of nodes stored in Neo4j makes this an easy capability to support.

Any time there are events associated with a resource, we keep a timeline of such events married to a snapshot of the associated resource(s) in the inventory at the time of the event occurrence.

Challenges

In the realm of NoSQL logging and reporting, consider the problems involved in monitoring a dynamic distributed environment with tools that are usually specific to a single technology. As Don put it, “what we need to understand is that the virtual and physical resources these products run on often overlap. At the very least there are server farms and networks that are shared”. NoSQL logging and reporting tools need to be able to identify patterns and relationships. They need an “elastic cross-technology solution that gets information on how [the technologies] impinge on one another in a common fabric”.

Another issue: in an environment where nodes are coming and going, a monitoring tool has to keep track of which nodes are current and which are not. As Don said “If we don’t get a report from a node, does that mean it’s just offline, or it has a problem? We also have to know at any given time in the history of our collection what nodes are available. Sampling a number of times helps you get a picture”. Sufficient samples over time can help ascertain whether a node’s state fluctuates a lot, with the caveat that maybe one can never be completely certain of even that. Maybe one rule could be that if a node is always ‘on’, and we get no reports on it for a [fill in the time frame], then we can conclude it has a problem; I think you see the challenge here.

Opportunities

Don understands the challenge and opportunity of NoSQL logging and reporting well; he says that “keeping the best snapshot” of a monitored resource is what we are striving to do with ECS, trying to “identify principal players” that a customer installation consists of, be they caches, nodes in a cluster, whatever they may be, in what is called our inventory. We can’t simply rely on current state, nor rely on history, but rather a combination of the two; “so that’s some of the stuff we’ve been looking at. If we can usefully compare what is in inventory now to what was there in the past, we’ll discover things we hadn’t even thought about, such as usage patterns and virtual and physical resources in that environment. I’m not sure we’ll be initially able to assess causality, but I think we can establish a footprint and allow the user to be able to explore and draw conclusions; I think we can give them the basis for that information.”

“It’s challenging to pinpoint cause and effect. For example, it’s difficult to determine that your publisher success rate is low because your CPU is maxed out; maybe your CPU is maxed out because you are attempting to do so much publishing that you’ve saturated that machine, leading to a low success rate; we can at least give them hints. We can also begin, with ECS 5.0, to give them projections, maybe presented graphically and in a number of visual perspectives; maybe an incident matrix.”

Regardless of what we deliver, Don says that “we want to give them something navigable, so they can begin to see where things are ‘lighting up’, then move distances away. So if a Cassandra cache was causing problems I could inspect the host. Oh, and now that I’m at that host, I can see there’s some other technology on there that’s beginning to have a lot of events.” Maybe this second technology is the actual root cause of the problem that first surfaced in the original monitored resource, a technology that is possible in a different tier, of a different NoSQL or caching technology, or maybe a servlet… you get the picture.

Don says “Being able to provide those hints and that navigation becomes even more important as the size and scale of these systems becomes such an issue that it becomes really difficult to monitor and manage the new environment without some event information or other heuristics that we’ve applied, a view that limits the scope of what they’re seeing to a space that we believe is related to problems that can explore causality. So, that’s one of the things we want to introduce in 5.0, some interesting visualizations, to help them navigate around.”

Weaving powerful semantics among tiers and domains based on an ever growing a better understood inventory of resources will provide a platform for discovering interrelationships whose understanding can well serve both root cause analysis and enforcement of SLA’s. This is the future of Evident ClearStone, as well as its present.

Learn more about our performance monitoring solution for Java, NoSQL and web servers

By Bill Nigh

NoSQL DB logging and reporting are challenging for a number of reasons:

  1. NoSQL is relatively new, so there are not a lot of experienced practitioners of the new technology, which most now understand to comprehend a wide variation of architectures; one definition of ‘NoSQL expert’ might be simply ’one who has product knowledge of one or more products’, as they are so different.
  2. Unlike RDBMS legacy technology, NoSQL DB logging and reporting cannot benefit from an ANSI standard common application and system query capability; while the RDBMS model exposes excellent system metrics via virtual ‘system’ tables that can be queried with the rich SQL language, this is not the case with NoSQL
  3. Single-point solutions are most likely the case, enabling a proliferation of monitoring and reporting consoles
  4. Entities participating in a cluster typically have separate logs on separate hosts in separate contexts, causing log correlation problems; for example cluster rebalancing, as found with Coherence, introduces load onto other nodes in that cluster, creating a ‘pathological’ situation and flurry of log entries that are not easy to interpret
  5. Collection methods can impact performance; an agent coresident in server memory with a NoSQL (or any) app, will impinge on the performance of the very process you are trying to monitor
  6. NoSQL is often part of a much larger system; e.g. Hadoop was a key part, but only part, of IBM Watson;  however, NoSQL logging and reporting tools are typically silos.
  7. The technologies mesh together in synergies hard to foresee and interactions that may not be anticipated; the performance implications of  all combinations of NoSQL running in the enterprise have not yet been documented, but a typical use case spans multiple NoSQL technologies
  8. A holistic view of NoSQL performance monitoring data is lacking, as each technology is associated with different metrics and (typically) different point solutions.
  9. There is so much data that one has to apply heuristics to filter out the valuable information, to avoid ‘drinking from a fire hose’
  10. Reporting such as found in JConsoleLINK does not persist report data for useful enough timeframes, so seeing trends and doing capacity planning are difficult
  11. The elastic nature of the environment means that as resources are deployed in response to increased demand, nodes will be coming and going, with life cycles that are not easy to predict; logging and reporting tools need to somehow account for the fact that a node may not report within a given time frame; does that mean it is just offline or it has a problem?

Bernd Harzog, writing recently in The Virtualization Practice, has furnished a very useful view of the state of the art and state of the industry here. It is well worth the time to read it (and I am proud to point out that he mentions Evident Software as an example of a company that is well positioned to make contributions toward addressing these challenges).

If you are interested in this subject, you may also want to read this post on challenges and opportunities in NoSQL DB logging and reporting.

Learn more about our performance monitoring solution for Java, NoSQL and web servers

By Bill Nigh

I’ve been speaking with Don Jeffery about ClearStone 5.0 and how its monitoring of NoSQL DBs can help with capacity planning. Don explains that once NoSQL resources are discovered through JMX or our open scripting-based API, administrators will be able to “see what resources are at their disposal, so that can see where they can introduce additional workloads.” Don says ClearStone’s monitoring infrastructure can readily support capacity planning, especially with the introduction of usage profiles.

“One of the things we’re also looking at doing is time on time comparisons, where we take sample metrics over one period of time and compare them to another period of time; perhaps that period of time we consider a benchmark; perhaps it’s just simply last week. As you begin to look at this and to trend it, you can begin to isolate things that can help you in your capacity planning; you can see, for eample, that the trend week over week is some monotonic increasing pattern of usage of some resource, so you budget some physical or virtual resources to cope with that.”

Capacity planning in NoSQL DBs is especially challenging because of the way the systems interact with other systems, so the typical point solutions one can have with each NoSQL or DCP product are not scoped appropriately. Also, the way the load is introduced into such systems can vary considerably over time. As Don has said “we’ve had customers that had tremendous difficulty at certain times of the calendar year simply because they’re doing something that involves, say, a promotion that introduces heavy outside load. It’s very useful to have these historical perspectives”. Don emphasized the importance of isolating the extraordinary from the expected, something our approach will support quite ably.

Learn more about our performance monitoring solution for Java, NoSQL and web servers.

By Bill Nigh

What special challenges are involved in pursuing useful NoSQL DB performance comparisons? Some of the challenges are: single point solutions, lack of a common query language/API to integrate metrics from different tiers into a common view, the heterogeneous nature of the technology, hard to predict lifecycles for objects such as nodes and caches, and other challenges, outlined here.

One approach to NoSQL DB performance comparison is to do time-on-time comparisons of single or multiple metrics with charts representing different monitored resources.  Don Jeffery puts it this way: “Depending on what your objective is when you are benchmarking, one of the things that you might do is to take a quiescent system, one that’s spun up but not incurring any load, and introduce some load to it over a period of time. For example, in Coherence, you establish the cluster, introduce, say, eight nodes, maybe eight JVMs over four loads, with caches introduced, but no clients.”

“You would monitor this quiescent system and inspect certain key metrics and expect them to be relatively flat. You’d then introduce loads and keep track of the times and loads introduced.” (Since we store history as time series data in Cassandra, getting this information for potentially long stretches of time is not an issue.)

Don continues: “If you keep track of the time and the load, then you can introduce time-on-time comparisons where you look at a number of key metrics. So now I have a benchmark; what I might want to do is vary that load in a certain way over a different time period, and perhaps compare against the initial benchmark and the previous set of measurements, to see if I can establish any kind of a trend. For example, I’ve attempted to increase the GET burden against the cache by 50%; does that translate into an equivalent translation of certain key metrics, or is there no linear relationship?”

We are about to introduce time on time measurements and thereby support this type of analysis, with some of the visualizatons we are planning in future releases of ClearStone. Multiple perspectives in the ClearStone real time dashboard, shared and customized, already support the presentation of charts from various resources in a free-form and easy to create manner.

Performance comparisons of different NoSQL DB technologies may be possible, Don said, if they are similar, say, two data grid technologies, with a deliberate and documented use of load and benchmarks. “You might look at response times, for example, or GETs, or look at cache hits.” This could enable a useful technology choice in situations where you know what kind, what size and what elasticity you anticipate in your data.

“Take the example of ehcache performance. The first consideration is to define what the key metrics are. One way to do that even with our system today is to establish reasonable thresholds and to use our thresholding policy tools to set those up and capture them and create events. That way we see if we violate any of those thresholds. We start with a quiescent system and introduce a load over an hour; during that time, ECS is running, so our collectors are doing their job. Line charts we assemble are being annotated with the events we’ve defined at the points of transition to the threshold.”

“We can do this with our product today, but not as conveniently as with 5.0 where we could use a single chart, we can do another test run and vary the load. For a data caching product, for example, we introduce additional caches or perhaps retune the network or cluster and start it up again with larger or smaller Java heap sizes or any number of other parameters we want to investigate; then we look at the behavior; then we run another load test; or perhaps same load, but vary the heap sizes. What we can do is to create a perspective in the real time dashboard; create two charts and compare them by using a metric that correlates the two time periods. What we hope to do soon is to have this comparison appear in the same chart. While we can do a better job of integrating the visualization, we can support those use cases today with two separate charts that are visually aligned in such as way as to allow comparison.”

Learn more about our performance monitoring solution for Java, NoSQL and web servers.

By Bill Nigh

SQL Performance Monitoring has a well understood model underlying some very mature monitoring and management tools. The RDBMS standard, of which SQL is a part, has a number of strengths that have earned it its central place in so many enterprises, features such as triggers, views, stored procedures, and declarative referential integrity. The salient RDBMS feature for SQL performance monitoring, however, is the presence, in deference to the RDBMS standard, of a large set of system tables, variously called the CATalog in Oracle, or the SYS-prefixed family of system tables in Sybase and MS SQL Server. The Relational Model, promulgated and championed by E.F.Codd, specified that a compliant RDBMS expose its internals to reporting using the same language and data model that was used for actual data. For this reason, even a bare bones command line interface to a relational database can, through SQL queries, harvest useful system data, thank Codd.

The performance monitoring area for NoSQL DBs differs in two major respects:

  • There are many products that come under that ‘elastic’ label; the Wikipedia article on NoSQL lists twelve types in its taxonomy (!)
  • As the name suggests, NoSQL has no SQL, no powerful and more importantly, standard language that can be used to query the system for performance metrics or other useful information about the resources found within the RDBMS, such as table size, number of indexes, percentage full of logical devices, and so forth.

NoSQL performance monitoring has to rely on tools that use an API, when present. Some NoSQL products have associated query languages, but, again, there is no common well known and well documented query language for metrics, putting a burden on organizations that have to monitor multiple NoSQL implementations.

This is where we come in.

Learn more about our performance monitoring solution for NoSQL, web apps and web servers: http://www.evidentsoftware.com

By Ivan Ho

As an Application Performance Management tool, Evident ClearStone collects and receives real-time data from disparate applications and systems in a distributed application environment. 90% of the data that ClearStone manages is the performance data from the application processes and systems. The other 10% are related to the inventory and events of the managed environment.

ClearStone 5 required an elastic data store that can handle all the scalability, performance, persistence, and flexibility requirements demanded of this class of product. The demands placed on this elastic data store ruled out the use of a traditional RDBMS backend.   The solution must have options for load balancing and high availability – being able to distribute and replicate data was essential. It was obvious that a NoSQL DB would come into play.

For example, in a single application environment with multiple servers, ClearStone would monitor system level metrics (CPU, disk, network, I/O) , application container performance, Java platform measurements, a distributed Cassandra cluster or other NoSQL clusters, and a database like PostgresDB.  The schemas of the performance data vary significantly from one type of resource to another, therefore we also needed a solution that would provide  flexibility in dynamic schema creation and updates while keeping the system running 24×7. At time when this was being developed, there was no one data caching, database system, or NoSQL solution that met all of these requirements. Ultimately, the Evident architects decided on using two NoSQL solutions, Apache Cassandra and Neo4j.

Cassandra is implemented as a time-series data store for storing all the real-time data and historical data. Our implementation uses Apache Cassandra 0.7 with the Hector client APIs for Cassandra. With Cassandra 0.7, we can dynamically create and evolve column families for storing all the performance data. The performance data is normalized by metric. We have also partitioned our column families based on the granularity of the data sets.

Neo4j is implemented as an inventory database used for maintaining all the managed resources (i.e. processes, hosts, clusters, etc.) of the application environment. It is used to store current state of all the resources, relationships among the resources, and correlated events to these resources. Anytime there are events associated with a resource, we keep a timeline of such events married to a snapshot of the associated resource(s) in the inventory at the time of the event occurrence. We felt the use of a graph database like Neo4j was ideal for storing metadata for the resources and mapping relationships and correlated events.

Beyond the performance and scalability benefits, we found that there were also some fringe benefits to each of these products. Both of these NoSQL systems have the options of being embedded into ClearStone or run as isolated clusters managed by ECS across multiple servers. Lastly both of these two products are cloud-friendly, therefore enabling ClearStone to be deployed in both traditional enterprise environment and cloud environments.

If you would like to learn more about our experience with either one of these NoSQL solutions, please come visit our Support Section and post your questions.

If you would like to try ClearStone, please join our Beta Program here.

By John Bennett

New Software Release Adds Operational Health Dashboard and Fine-Grained Alerting

ASBURY PARK, NJ, December 14, 2010 — Evident Software, the leading provider of Application Performance Management (APM) for NoSQL, data caching, and Java applications, today announced the availability of Evident ClearStone® 4.6, the company’s production-class solution for monitoring and managing all tiers of enterprise applications. ClearStone 4.6 introduces new features to help development, operations, and support teams monitor and troubleshoot business-critical applications. Using ClearStone 4.6, engineers can correlate the status of JBoss or WebLogic application servers with other data tiers, such as a Cassandra NoSQL DB tier or system-level metrics. This multi-tier correlation helps engineers rapidly identify the root cause of performance problems.

New Features Improve Visibility and Accelerate Troubleshooting

ClearStone 4.6 introduces an Operational Health Dashboard that enables engineers to discover at a glance the health of application resources. ClearStone determines each resource’s health based on system metrics, key performance indicators (KPIs), and the status of events. The Operational Health Dashboard presents green, yellow, and red indicators of a resource’s health over time. Support engineers can click on a health indicator to discover the underlying details and to read troubleshooting tips written by the applications developers. By putting this information at a support engineer’s fingertips, ClearStone 4.6 accelerates troubleshooting and helps bridge the divide between Development and Operations.

ClearStone 4.6 also introduces fine-grained alerting that enables specific users or groups of users to be notified when specific conditions occur. This fine-grained alerting helps developers and operations teams ensure that the proper people are notified when specific error conditions or outages occur.

“With the release of ClearStone 4.6, we’re giving developers and operations engineers unprecedented visibility into enterprise and Web applications, including ‘Internet-scale’ applications that work with vast amounts of data,” said Scott Barnett, CEO of Evident Software. “ClearStone lets developers and operations engineers see all tiers of an application, now including the JBoss or WebLogic application server tier, in a ‘single pane of glass.’ This comprehensive view of real-time and historical data gives IT departments the information they need in order to optimize application performance.”

Web Seminar with Sneak Preview of ClearStone 5.0

Evident Software will present a Web seminar on “The Present and Future of Application Performance Management for NoSQL and Data Caching Architectures” at 2 pm EST on Wednesday, December 15, 2010. To register, please visit: http://www.evidentsoftware.com/category/news-events/events/.

About Evident Software, Inc.

Evident Software delivers the first comprehensive application performance management solution for NoSQL, data caching and Java applications. The company’s ClearStone platform enables developers and operations personnel to monitor, manage, and optimize business-critical and Internet-scale applications. Evident’s solutions are installed in the financial, SaaS/cloud, e-commerce, government and IT services industries. The company is based in Newark, N.J. with research and development facilities in Asbury Park, N.J.

###

Evident ClearStone is a registered trademark of Evident Software. All other trade names are the property of their respective owners.

For media inquiries, please contact:

John Bennett (for Evident Software)
john@bennettstrategy.com
+1 510-495-6590

By Scott Barnett

DevOps might just be a buzzword to some people, but we’ve been seeing the need for integrating development and operations firsthand. Customers tell us they start off needing ClearStone (our monitoring and management platform for NoSQL DBs, data caching, Java and web applications) in development and testing, sometimes many months before an application makes its way into production.

In the past, the idea of developers using ClearStone presented a bit of a challenge for us, because our business model was developed with production applications in mind.  But as we’ve continued talking to prospects and customers, it’s been clear that we need not only to make ClearStone affordable and accessible to developers, we need to continue to innovate and add features specifically for developers (features like cache browsing, load testing, management automation, integration with build/deploy tools). Whether or not the operations team uses ClearStone, developers need ClearStone’s real-time monitoring, detailed analytics, and rich visualizations.

We are proud to announce our first official product for developers: the ClearStone Development Pack. 

We have been quietly offering Development Packs for about a month and are now ready to formalize the product.  A Development Pack is sold on a per-developer (per-seat) basis.  This is a fully functional version of ClearStone – and it can be run on an unlimited number of servers as long as they are not production servers.  This allows a developer to use ClearStone on their own workstation for unit testing, but also on their Q/A and staging servers for load/stress testing, capacity planning, and active/passive monitoring in development.  Once the application is ready for production and onto production servers, customers will be required to purchase production licenses of the product.  The production pricing model will be changing too in early 2011, but I’ll save the details about that for another post next month.

The Dev Pack will be priced at $995/user/year – and comes with email support, free upgrades/updates to the software, and access to the ClearStone wiki.  To celebrate our launch, we’re offering a 50% discount on Dev Packs through the end of December – so (or more!), get the value of ClearStone in development, and contact us to learn how to best leverage ClearStone in both development and then production. 

We’re confident you will find the tool more than valuable for the price, and look forward to working with you and your team.  Please be sure to tell your friends as well!

By Scott Barnett

Last month, we launched Evident ClearStone 4.5, which includes NoSQL logging and NoSQL reporting features. This software release marks an important milestone in the evolution of Evident Software. Over the summer, we made a decision to aggressively go after the NoSQL DB market, expanding our previous support for compute-grid technologies such as DataSynapse and application-grid technologies such as Oracle Coherence by offering NoSQL reporting, NoSQL logging, management and performance monitoring.

Why make this change of course? There were several reasons:

  1. NoSQL might not still be called “NoSQL” in a few years, but it absolutely will be an important technology for enterprise applications. Think back to how the Java Application Server came of age in the mid/late 1990′s. That technology required several iterations to become the Java Application Server. The market took a few years to coalesce and turn into something that the broad industry could understand, market, and build around. Today NoSQL is going through a similar evolution. We’re just starting to see forecasts of the size of the NoSQL market. We suspect it won’t be called NoSQL a year from now (some other people seem to agree) – as technologies such as Hadoop, Data Caching Platforms such as Coherence, GemFire, Terracotta and hybrid in-memory databases such as VoltDB all vie for developer mind-share. Whatever it’s called, this is the “new” tier in the application stack, and it’s going to need focused and dedicated capabilities from a management/monitoring perspective, including NoSQL logging and NoSQL reporting. Here’s a great database (no pun intended) of systems that fall into the NoSQL realm.
  2. Correlating metrics and events between the NoSQL tier and the other existing tiers in the application (and system) stack will be key capabilities for monitoring and managing NoSQL applications. Each tier cannot continue to have its own NoSQL logging and monitoring capabilities – monitoring needs to be integrated, so enterprises can get a holistic view of their applications. This is a hard problem to solve.It’s also a valuable problem to solve. We are solving this problem already now for the caching technologies I listed above. Now we want continue extending this capability across the different tiers of the application stack.
  3. Visualization is the key to success in APM. When you are gathering so many metrics/events in real time, it’s a challenge to determine what is really important to DevOps.. We’ve been told we’ve done a great job of figuring this one out – our user interface is intuitive, attractive, and meaningful. Making sense of all that data is hard to do. Without it, you have lots of great data with no insight. You need insight to make good decisions.
  4. Our goal is to support every NoSQL system out there. To meet this goal requires a change of strategy – so you will see us open up our platform so that people can build and deploy their own “Management Packs.” We currently have Management Packs for DataSynapse GridServer, Oracle Coherence, Apache Cassandra, Memcached (with Membase coming very soon), WebLogic Server, and jBoss. We are working on many more, but we want to move even faster. So you will see a Management Pack framework that allows developers to build their own Management Packs (we can help you too!). It is not hard to do this, and we will roll out a developer site shortly for people to share/collaborate/contribute. We will start by contributing our own management packs to the site.

So, 4.5 is the next step in our evolution and a hearty step forward in our embrace of all things NoSQL as the latest, greatest participant in the application stack. From our conversations with customers and prospects over the past few months, we know many of you agree our vision of NoSQL reporting, monitoring and management. We look forward to working with you on this initiative in the months and years ahead. We are very interested in your thoughts, ideas, and suggestions on how to continue this process, so please share your ideas with us!