Knowing if performance is getting better, worse, or staying about the same – it can have a big impact on management decisions. The tool that helps us understand how things are going is also one of the most important data visualization tools in the quality management toolbox – the run chart. This vlog will explain the basic elements of a run chart and how often to add new data points. This is the first video in series that will look at run charts, understanding the variation you see in a run chart, and then, how to interpret the run chart to make management decisions. (Duration: 8 min. 9 sec.)
Notes and Resources
There are lots of resources online to learn more about run charts. With this one, I tried to put it in context of systems of care for HIRTSCs (High Risk Time Sensitive Conditions). Below is a link to an excellent into to run charts video that looks in a bit more detail on the technical aspects of building a run charts. Another link below show how to build a run chart using Excel
- Run Charts in Quality Improvement – (7 min 13 sec) https://www.youtube.com/watch?v=ySbhsX-y8zE&list=WL
- Building a Run Chart in Excel – This is a very brief video (1 min 13 sec). Pretend the sample 1 through 5 data in this video represents the five STEMI receiving center hospitals in the community I described and that the averages are for the monthly data point that shows the symptom onset to device time interval on STEMI cases for the entire community. https://www.youtube.com/watch?v=0FNzoB19G4A
How often have you attended a meeting or read a report where a number is presented that indicates what the performance of something was for the current month – just that one number alone. Depending on the context, that might be OK. But in the scenarios I’m thinking of, the target audience was being informed about how well things were going regarding a process, program, or the overall organization. If everybody in that target audience is well informed about how things have been going in the past for quite some time and know specifically what the numbers have been, that may also be OK. But in my experience, that’s usually not the case. Without knowing what the number was for the previous month, it’s difficult to impossible to know if that one number being reported is higher or lower. It’s little bit better when a comment is made that this one number is higher or lower than the previous month. Even better, the specific number from last month is stated. Now you can calculate in your head how much of a change there is. Even better yet, both numbers are given and the presenter or report does the math for you – maybe you’re even told what the percentage of change is from the previous number. Regardless, it is hard to really make much of an assessment on how things are going with just two numbers. This is, sadly, a very common problem. So much so that we have a tool specifically designed to remedy this problem. It is one of the most basic and important data visualization tools we have in the quality toolbox – it’s called a run chart.
The run chart is the graphical display of a set of data points that are shown in the order they were collected over a period of time. The customary way of showing the data points is with the value of the number being reported being on the vertical axis, also called the Y axis. Usually, the lower values are at the bottom and the higher ones reaching upwards. Time is on the horizontal axis, also called the X axis. Usually, the most recent point in time is to the right side of the graph and the oldest data is on the left side.
Let’s look at an example. Suppose we have a community where they have a formally designated STEMI system of care. They formed a consortium of all the major players that deliver the emergency care. It has all of the STEMI receiving hospitals that do cardiac caths and both the ED and the cardiac cath lab teams are represented; ED representatives from all of the referral hospitals that send STEMI patients to the receiving hospitals are there; all of the ambulance services and non-transport medical first response agencies have a rep there; emergency medical dispatch centers that manage STEMI patients before hospital arrival or during inter-facility STAT TRANSFERS from the referral hospitals to the STEMI receiving hospitals are all represented. This acute STEMI care consortium meets every month to review performance data on various aspects of their STEMI system of care.
One of the key performance indicators they focus on is the symptom onset to device time. It’s their best approximation of total ischemia time – which is how long an acute episode of myocardial ischemia lasted before the coronary artery was opened. Research shows that the longer that the period of total ischemia lasts during a STEMI, the more of the myocardium is lost and the worse the outcome. Time is muscle. The consortium is trying its best to do things to reduce that symptom onset to device time. So, they need to know if the performance of the system is getting better or worse. They need to know if the changes they have made to the system of care are making an impact – or are things staying about the same. They all submit data to a clinical registry database which collects and analyzes information from all of the STEMI cases at all of the STEMI receiving centers. Shortly after the end of each month, the elapsed time on each STEMI case from acute symptom onset to the opening of the artery in the cardiac cath lab is calculated. All of those elapsed time values are put together to calculate an average. A run chart is then generated from that system-wide average result. The data points are connected with as simple line. Looking at their last three years of data with monthly data points, we get this run chart. Being able see the data in this graphic format, a data visualization, gives us a much better idea of how things are changing over time throughout the region that the consortium serves.
One of the other issues with run charts is how often to add a new data point. We often think in terms of each day, week, month, quarter or year. In general, I suggest you add new points as often as you have enough cases to be representative of how well the process overall is working. Just 1 or 2 is cases is not really enough. Hundreds is much more than needed. 10 or so is probably a good place to start. So if you consistently have around 10 or so cases a week, I’d add data points weekly. If it takes a month to accumulate that many cases, then add a data point monthly, and so on. In a separate post, I’ll talk about how you might address this challenge in low case volume scenarios. The advantage of having new data points added more often is that you get to detect changes in performance sooner than later. If something is going wrong, it’s better to know as soon as possible so you can intervene. If something is getting better, you can try to lock in that change as soon as possible to make sure that progress is sustained – and maybe spread it to other places. It is bit of trade-off. Larger groups of cases for each data point adds some precision to the picture of how things are performing, but you do not get to see changes as quickly. Smaller groups do not give as precise of a picture of how things are going, but you can detect any changes sooner. So, 10 or so cases per each time period can give you a nice balance. It’s OK that some time periods have a few more and other time periods have a few less. If the average per time period is around 10, you’re probably in pretty good shape.
You can also consider adding a new data point based on a fixed number of new cases. Instead adding a new data point at regular time intervals like each week or month, you add a new data point with each set of new consecutive cases. You might choose 10 cases for calculating each data point. Once you decide on how many cases per data point, stick with that number of cases per data point. This approach may be particularly helpful when you have seasonal swings in the number of cases.
With your run chart, you get a clear view of how things changing are going over time. It lets you see if things are generally getting better, worse, or staying about the same. These types of assessments get us into the realm of trends. What exactly is a trend? How much of changes] is needed over time to be considered a trend? Those are extremely important questions, as it will influence your interpretation of the run chart and influence your management decisions about how to react to the information on the run chart. We’ll take up the issue of assessing trends in the next post.