Type 2 Diabetes Remission: Low Carb versus Low Calorie, and the difficulty with comparing studies
Dr Sean Wheatley, MSc, PhD. Science and Research Lead
Type 2 diabetes was previously thought to be a progressive condition, but it is now clear that this need not be the case. Ever-increasing evidence shows that remission is possible for many people, with low carbohydrate and low energy (i.e., low calorie) approaches amongst the best supported methods for achieving this.
We have previously blogged about the subject of remission (see here and here), and also have a short information leaflet on the subject which can be accessed from our Knowledge Hub. This blog adds to these by taking a look at the longer-term results of two of the most important studies in the area, as well as exploring challenges with comparing the two.
The Studies
Although evidence comes from a wide range of sources, some of the best comes from DiRECT and Virta Health.
DiRECT used meal replacement soups and shakes to help people follow a low energy diet for a short(ish) period of time. This usually meant having around 800 calories per day for between two and six months.
Virta support(ed) their participants to adopt a very low carbohydrate dietary approach. In most cases this involved having 30g carbohydrate per day or less.
More details of these approaches/studies can be found in the blogs linked to above and/or amongst the plethora of published papers (see here and/or here), but the most important point is that both of these methods were shown to be effective for helping some people to achieve type 2 diabetes remission, in the short-term at least.
Five year results have now been published by both DiRECT and Virta, providing an opportunity to see how participants have fared longer-term. Before comparing their outcomes though, it is important to consider differences between these studies to make sure we can consider their results in the appropriate context.
Challenges with Comparing 1 – Defining Remission
One of the first difficulties with comparing the results is down to definitions. Remission of type 2 diabetes is usually classed as having a HbA1c (a marker of blood glucose control) below 48mmol/mol (the threshold used for diagnosing type 2 diabetes) for a period of at least three months, whilst not needing any blood glucose-lowering medication to achieve this.
In DiRECT, the aim was specifically to achieve diabetes remission. They therefore set out to withdraw all diabetes medications, to try and meet this definition.
Virta however took a different approach, as they did not take their participants off Metformin, a common diabetes drug, unless it was causing them any issues. This means some of their participants could not meet the standard definition of remission, and we don’t know if they would have if metformin had been removed.
This difference in approach immediately creates a barrier to being able to compare results between the two studies.
Challenges with Comparing 2 – Participant Characteristics
DiRECT was quite strict with limiting inclusion to participants who were considered to be more likely to achieve remission, based on findings from preliminary research in the area. This meant that people who were taking insulin and/or people who had been diagnosed with diabetes more than six years before the start of the study were not allowed to take part.
In contrast, Virta did not apply such strict limits. In fact, nearly 30% of Virta participants were taking insulin at the start of the study, and the mean duration of diabetes at baseline was 8.4 years. In any direct comparison of remission rate, Virta were therefore significantly hamstrung, as they included lots of participants who would be considered less likely to achieve remission.
Challenges with Comparing 3 – Other Study Characteristics
As well as differences in who was allowed to take part, there were other important differences in the nature of the studies which could have impacted the outcomes.
One of the biggest differences is that in DiRECT participants were randomised to either the intervention group (the low energy diet) or the control group (“usual care”), whereas in the Virta study there was no randomisation. Randomised trials are generally considered to be higher quality, as they reduce the risk of certain biases influencing the outcomes. Non-randomised studies are not without their benefits though, such as that they are often a better reflection of a “real world” setting. There are pros and cons to each, but this fundamental difference could potentially influence the results.
Other differences include the nature and quantity of guidance and support provided to participants. This can affect whether people stick to an approach and/or whether they follow it “properly”, independent of the approach itself. For example, when someone drops out of a study it might be because of the intervention (e.g., they don’t like the diet or they might think it is not working) or it might be because of some other element of how the study is being run (e.g., they might not have been given clear guidance on what they should be doing and/or sufficient support to keep them interested and motivated). This is difficult to unpack, which further complicates the ability to interpret outcomes, particularly when comparing between studies.
The important point here is that not all differences in the results will be because of what is supposed to be being studied! It is therefore overly simplistic to present the results as a direct head-to-head comparison of a low carb and a low energy approach.
Challenges with Comparing 4 – Presentation of Information
The next challenge is in relation to how the results are presented. There are dozens of different ways to put forward research findings, which can have a significant impact on how they are interpreted. There are legitimate debates as to the best method(s) to use, with pros and cons to a range of different approaches. But sometimes there are deliberate attempts to try and mislead people by making certain choices too. As Mark Twain once (apparently) said, “there are lies, damned lies, and statistics”.
Two of the most common methods for presenting results from studies like DiRECT and Virta are:
1. On an “intention to treat” basis.
This approach presents the results based on everyone who started a study. So, if you have 100 people at baseline, you would report how many of your 100 people achieved the outcome in question by the end of the study.
This approach is conservative, as it essentially treats any dropouts or people with missing data as being people for whom the intervention did not work. This avoids overinflating results.
2. On a data-availability basis.
This method involves presenting results only for participants who have relevant data available at the relevant time point(s). It excludes all dropouts and people with missing data. This has the benefit of providing an indication of how well an intervention is likely to work if someone completes it (assuming the “completers” followed the protocol properly, but that’s a separate issue entirely!).
This approach can risk exaggerating the effectiveness of the approach being tested though, as it ignores the fact people may have dropped out because the intervention was not working and/or because they could not stick to it. In general, people are more likely to drop out of a study if one of these things is true, so excluding them from the final analyses can introduce a source of bias.
There are lots of other options, providing ample opportunities, deliberate or accidental, to massage or inflate (or downplay or hide) certain results. One such example can be found in the DiRECT 5-year results paper, where the remission rate is initially presented as being a pretty impressive 26%. Many people will go no further than the numbers presented front and centre by the study authors, and/or will not take the time to consider where they are derived from. As a result, these headline numbers are often the ones which are shared, with important context lost in the process. In this case though, this value was based only on those who were already in remission after 2 years, ignoring 101 people who were not. It can be argued (and I would) that as a headline figure this is misleading, as it gives the impression that the approach was more effective than it was (compare this to the results I consider to be a better reflection of the outcomes in the next section, and see what you think).
The “best” and most appropriate method(s) could be debated almost endlessly, but the key point is that the numbers used can significantly impact the conclusions. This is important to bear in mind when trying to interpret studies.
DiRECT versus Virta – Attempted Comparison
So, back to the actual studies! Taking the results that are most comparable, so we are comparing “apples with apples” (as far as is possible, whilst not forgetting the key differences already discussed), the key figures from these two studies are (in my opinion) as follows:
DiRECT:
– 11 of those still involved in the study after 5 years were in remission at this time point.
– The five-year remission rate for everyone who completed the study and for whom relevant data were available was therefore 13% (11/85).
– On an “intention to treat” basis (based on everyone who started the trial) the remission rate was 8% (12*/149).
– Of those in remission at 2 years who were still involved in the trial after 5 years, remission was maintained by 23% (11/48) on an intention to treat basis, or 26% (11/43) if only those with available data at both time points were included.
Virta:
– 24 of those still involved in the study after 5 years were in remission** at this time point.
– The five-year remission rate for everyone who completed the study and for whom relevant data were available was therefore 20% (24/120).
– On an “intention to treat” basis (based on everyone who started the study), the remission rate was 9% (24/262).
– Of those in remission at 2 years who were still involved in the study after 5 years, remission was maintained by 46% (19/41) on an intention to treat basis, or 58% (19/33) if only those with available data at both time points were included.
Direct versus Virta – Interpreting the Results
So, the intention to treat results are, superficially at least, actually very similar (8% remission in DiRECT versus 9% remission for Virta). The question then, is how to interpret this within the context of the challenges discussed before. This is a process during which it is essential to try and manage any biases you have.
My interpretation is that these numbers favour Virta, based on how differences in the study protocols and/or participant characteristics could bias the outcomes “against” Virta. That is to say:
1. Some of the participants who did not meet the remission criteria may have done if their metformin had been deprescribed – i.e., the “failure” to achieve remission may in some cases be due to the methods used during the study rather than the impact of the intervention. The results presented may therefore be an underestimation of the very low carb dietary approach’s efficacy.
2. Compared to DiRECT, the longer mean duration of diabetes amongst the participants and the inclusion of participants who were taking insulin means that the chances of achieving remission were probably lower for the Virta cohort. That a comparable remission rate was achieved despite this suggests that the Virta intervention was actually more effective (i.e., the “success” rate would have been expected to be lower, but it wasn’t).
You could however argue that the randomisation used in DiRECT makes it higher quality evidence than that provided by Virta, thus afford the DiRECT results a greater weighting. Even then, whether this outweighs some of the other relevant factors is still a judgement call!
As for the outcomes when considering only participants for whom relevant data were available at the 5-year timepoint, the results suggest the Virta intervention was more effective (even before trying to factor in the additional considerations outlined above), with a remission rate of 20% compared to 13% for DiRECT.
Maintenance of remission also seemed to be better for Virta, based on the percentage of people who were in remission at 2 years who were still in remission after 5 years (46% or 58% for Virta, and 23% or 26% for DiRECT, depending on what method is used).
Again, these outcomes should all be interpreted in the appropriate context.
So, What’s the Bottom Line?
The most important thing is that both approaches were effective for some people. Growing evidence that remission of type 2 diabetes can be achieved and sustained provides hope to millions of people with the condition, and this should not be lost amongst arguments over what the “best” approach is. Ultimately, one size does not fit all. Providing a menu of options – then supporting individuals to identify, adopt and adapt an approach that is suited to THEIR needs and preferences – is the best way to help people meet THEIR personal goals***.
After that, the key take home here is that interpreting research results can be challenging, particularly when trying to compare studies that have fundamental differences. Trying to consider the appropriate context, and trying to be mindful of your own biases, is important. MY interpretation here is that the Virta outcomes were better than the DiRECT ones. I’m sure not everyone will agree!
* You will (hopefully) have noticed that there is a 12th case of remission included here. This is because there was one confirmed case of remission at 5 years amongst the people who had completed the initial 2 years of the study but opted against participating in the follow up period. It is only fair to count them as a successful case in an “intention to treat” analysis, as this method counts everyone, and it is verifiable that the desired outcome was achieved for this person. It would therefore be unreasonable (I think) to count them as a non-remission case. They cannot be counted in the other method I have presented though, as they are not a “completer” of the study. This further demonstrates the potential complexity of deciding how to present and interpret data!
** These are the figures for “partial remission” in the Virta research paper, which corresponds to the definition of remission used in DiRECT (and that is outlined earlier in the blog). Virta also include figures for “complete remission”, which requires an even lower HbA1c, and “reversal”, which includes people who were still taking metformin. The numbers for these latter two definitions are therefore not directly comparable to the figures from DiRECT.
*** Other evidence, not discussed here, suggests that ANY approach which can help someone lose enough weight, and keep it off, can help them to achieve remission of type 2 diabetes. It isn’t therefore a straight choice between these two options outlined in this blog, these just happen to be two of the best examples we have right now that this is possible.
