Abstract
Electrolyte and acid-base disorders are commonly seen in critically ill and other hospitalized patients. A scoring system is needed to assess the severity of electrolyte and acid-base disorders and to predict outcome in hospital patients. Herein, we prospectively enrolled a total of 322,046 patients, including 84,700 patients in the derivation cohort and 237,346 in the validation cohort, in a large, tertiary hospital in East China from 2014 to 2017. A points-scoring system of general electrolyte and acid-base disorders with a sum of 20.8 points was generated by multiple logistic regression analysis of the derivation cohort. Receiver operating characteristic curve analysis showed that the optimal cut-off value of 2.0 was associated with 65.4% sensitivity and 88.4% specificity (area under the curve: 0.818 (95% CI 0.809 to 0.827)) to predict hospital mortality in the validation cohort. On Kaplan-Meier survival analysis, the five intervals of risk score (Q1: 0 to 2.0; Q2: 2.1 to 2.5; Q3: 2.6 to 3.3; Q4: 3.4 to 4.5; and Q5: >4.5 points) showed differences in hospital survival (p<0.001). Elevated (delta) risk score >2 during hospitalization increased the risk of hospital death, while those with a delta risk score <0 and <−2 points had higher survival rates. This novel scoring system could be used to evaluate and to dynamically monitor the severity of electrolyte and acid-base disorders in hospitalized patients.
Significance of this study
What is already known about this subject?
Electrolyte and acid-base disorders are commonly seen in critically ill and other hospitalized patients.
Patients with mixed electrolyte and acid-base disorders have a worse prognosis than those with either of these conditions separately.
Up to now, there is no method to access the general status of electrolyte and acid-base disorders, and the associations with hospital outcomes have not been revealed.
What are the new findings?
Herein, we conducted a prospective cohort study and generated a points-scoring system of electrolyte and acid-base disorders with a sum of 20.8 points by multiple logistic regression analysis in the derivation cohort.
This risk score had an excellent performance with an optimal cut-off value of 2.0 (sensitivity 65.4% and specificity 88.4%) and area under the curve 0.818 (95% CI 0.809 to 0.827) to predict hospital mortality in the validation cohort.
Kaplan-Meier survival analysis also showed its excellent significance in dynamically monitoring electrolyte and acid-base disorders.
How might these results change the focus of research or clinical practice?
This novel and dynamic risk score is reliable to evaluate and to monitor the severity of electrolyte and acid-base disorders.
Introduction
Electrolyte and acid-base balance is crucial for the maintenance of homeostasis in the body and plays a critical role in protecting cellular function and tissue perfusion. Electrolyte and acid-base disorders are commonly seen in critically ill and other hospitalized patients. The presence of these disorders typically reflects the development of underlying pathology and is associated with poor outcomes.1–3 Notably, patients with mixed electrolyte and acid-base disorders have worse prognosis than those with either of these conditions separately.4 Up to now, there is no method to access the severity of overall electrolyte and acid-base disorders and to reveal the association with poor outcomes.
So, it is essential to establish the overall frequency of distribution and combination of these disorders; however, previous studies have mostly focused on the disorders of specific electrolytes and acid-base abnormalities5–7 or have evaluated the overall status in surveys with small samples.8 Due to the complex mechanisms and a shortage of relevant studies, the analysis and management of electrolyte and acid-base disorders remain a challenge for physicians. Recently, with the development of electronic databases, it has become possible to transfer data from electronic medical records into research work and to describe the overall distribution of electrolyte and acid-base disorders. However, even if the prevalence of these disorders in various situations has been fully revealed, measures for prompt awareness and appropriate correction of electrolyte and acid-base disorders remain unsatisfactory.9 10 For example, we reported in our previous study11 that hyponatremia was associated with high risk of hospital mortality (OR 2.225, 95% CI 1.857 to 2.667), but the OR increased to 56.884 (95% CI 35.098 to 92.193) if hyponatremia was overcorrected to hypernatremia. Therefore, the eventual goals of current research should be to evaluate the effects of overall electrolyte and acid-base disorders on the prognosis of hospitalized patients, and to monitor (especially electronic alerting and monitoring) and correct them appropriately and in a timely manner. Currently, however, there is no such evaluation system for electrolyte and acid-base assessment that can help achieve the abovementioned healthcare aspects. So, the aim of this study was to develop and evaluate a novel scoring system to monitor electrolyte and acid-base disorders and to guide the appropriate corrective measures.
Materials and methods
Design overview
In the period from October 2014 to September 2015, we prospectively collected 1-year data pertaining to electrolyte and acid-base status of inpatients at a single center as the derivation cohort, examined the prevalence of electrolyte and acid-base disorders during hospitalization, and, most importantly, established a novel and dynamic risk score for the assessment of electrolyte and acid-base disorders. And then we collected 2-year data of the patients admitted from January 2016 to December 2017 as the validation cohort. Herein, the utility of this risk score to monitor disease severity and to create an electronic alerting system for these disorders was assessed, with the aim to predict poor outcomes and, ultimately, to guide the correction of disordered parameters.
Study population
The study population comprised all subjects admitted to Zhongshan Hospital, Fudan University, Shanghai, China—a tertiary hospital—from October 01, 2014 to September 30, 2015 (derivation cohort), or from January 01, 2016 to December 31, 2017 (validation cohort). Patients younger than 18 years and those without any data on electrolyte and acid-base status were excluded.
All data were collected from an electronic medical records database. These data included demographics, diagnosis, and laboratory values at admission. All results of measurements of electrolyte and acid-base levels during hospitalization were obtained. Moreover, hospital mortality and incidence of acute kidney injury (AKI) were recorded as the outcomes. As this was an observational survey, the requirement for informed consent was waived.
Definitions and calculations
Serum sodium, potassium, chlorine, calcium, phosphorus, and magnesium levels were measured in the clinical laboratory department of the hospital using standard methods on an automatic biochemical analyzer (Hitachi 7600, Japan). In this study, standard normal ranges at the hospital for serum sodium, potassium, chlorine, calcium, phosphorus, and magnesium levels were 137–147 mmol/L, 3.5–5.3 mmol/L, 98–110 mmol/L, 2.15–2.55 mmol/L, 0.8–1.45 mmol/L, and 0.67–1.04 mmol/L, respectively. Values beyond these normal ranges were considered indicative of electrolyte disorders. Because the serum albumin level affects the total calcium level, the corrected calcium level was calculated using the formula:
Corrected total serum calcium level (mmol/L) = measured total calcium level (mmol/L) + 0.02 × (40 − albumin concentration (g/L)).
The anion gap (AG) was calculated by the formula:12
with an elevated AG of greater than 16 mmol/L. AKI was defined by the Kidney Disease: Improving Global Outcomes creatinine based criteria as:13 14 an increase in serum creatinine (SCr) >0.3 mg/dL (26.5 µmol/L) within 48 hours, or an increase in SCr to >1.5 times the baseline known or presumed to have occurred within the preceding 7 days. Arterial blood gas values were measured by Medica Easy Electrolytes (Medica Corporation, Bedford, Massachusetts, USA). Simple, dual, and triple acid-base disorders were diagnosed according to the flow diagrams described by Milford Fulop.15
Statistical analyses
SPSS V.24.0 (SPSS, Chicago, Illinois, USA) was used for all analyses. A p value <0.05 was considered statistically significant. In addition to p values, standardized differences were calculated for each variable. It is used to distinguish clinical from statistical significance in large samples, and the threshold of 10% or greater is recognized as a clinically important difference.16 All continuous variables are expressed as median (IQR) as most of these variables were non-normally distributed. These data were analyzed using the unpaired t-test, analysis of variance, least square difference test for post hoc comparison, Mann–Whitney U-test, or Kruskal-Wallis test, depending on the variable distribution and the number of groups. Categorical data are presented as frequencies and percentage, and between-group differences assessed with the χ2 test.
Multiple logistic regression models with the Wald forward stepwise method were constructed to assess the association between the most severe electrolyte and acid-base disorders during hospitalization and hospital mortality in the derivation cohort. All these indices of disorders were transformed to categorical variables, that is normal, mild or severe groups according to normal ranges, clinical experience, and based on other published data. In addition, we also referred to 5% of the abnormal values to choose the cut-off values between mild and severe disorders. For example, the patients with less than 130 mmol/L and over than 150 mmol/L accounted for about 5% of patients with dysnatremia. Then, a score-based prediction rule was developed on the basis of the results of the multivariable logistic regression by using a regression coefficient based scoring system. Scores were assigned by reserving one significant digit after the decimal point of the regression coefficients if the p value was <0.05. The overall risk score was defined as the sum of scores for each component.17–19
Finally, the prognostic significance of this risk score on hospital mortality was estimated in the validation cohort. The ability of this risk score to predict hospital mortality was assessed by receiver operating characteristic (ROC) curve analysis. The optimal cut-off value was defined as the point that maximized the Youden Index, defined as ((sensitivity + specificity) − 1).20 The risk score was then divided into five intervals, including the interval with scores less than the optimal cut-off value and four according to the quartiles with scores more than the optimal cut-off value. Cox proportional hazards models with the Wald forward stepwise method was undertaken to assess the association between the risk score and hospital mortality adjusted for age, sex, body mass index (BMI), diastolic blood pressure, systolic blood pressure, and fasting blood glucose (FBG), albumin, hemoglobin, white blood cells (WBCs), SCr at admission as well as the incidence of AKI. After that, the severity of electrolyte and acid-base disorders at admission, during hospitalization, and before death or discharge was re-evaluated by the risk score, and the increased risk score (delta score) was calculated. Kaplan-Meier survival curves stratified by the risk score and delta score were plotted and compared using a signed log-rank test.
Results
In total, we enrolled 99,847 and 295,252 hospitalized patients, respectively, as the derivation and validation cohorts. After excluding patients younger than 18 years and those without electrolyte measurements, a total 322,046 patients, including 84,700 patients in the derivation cohort and 237,346 in the validation cohort, were included in the final analysis. Data pertaining to serum sodium, potassium, chlorine, carbon dioxide combining power (CO2CP) and AG levels were available for all patients. Data pertaining to serum calcium, phosphate, and magnesium levels were available for 57,904 patients in the derivation cohort and 153,484 in the validation cohort, while results of arterial blood gas analysis were available for 12,438 and 44,192 patients, respectively.
Baseline characteristics
Baseline characteristics of the study population are listed in table 1. The median age was 59.0 years, and 60.9% were male in the derivation cohort, while the median age was 60.0 years and 60.8% were male in the validation cohort. Hematologic and oncologic diseases followed by cardiovascular and respiratory diseases were the most common admission diagnoses in both, the derivation and validation cohorts. The prevalence of AKI was 11.8% and 10.5%, respectively, in the derivation and validation cohorts, which was significantly different (p<0.001). The incidence of AKI at stage 3 in validation cohort was also substantially lower than that in the derivation cohort (0.7% vs 1.4%, respectively, p<0.001). There was no statistically significant difference in hospital mortality with 1.1% and 1.2% in the derivation and validation cohorts (p=0.292), respectively. The standardized differences of each variable are also shown in table 1.
Prevalence and outcomes
As shown in table 2, the top three common electrolyte disorders were hyponatremia (16.6%), hypokalemia (13.7%), and hypochloremia (13.6%). Patients with hypernatremia experienced the highest mortality (11.0%), which was followed by hyperkalemia (10.0%). Acid-base disorders were found in 62.9% (n=7760) patients with blood gas analysis. Simple, dual, and triple acid-base disorders were seen in 42.1%, 16.6%, and 3.7% patients, respectively, and the mortality rates among these patients were 4.2%, 6.5%, and 10.5%, respectively. Approximately 87.4% of the 7760 patients with acid-base disorders had associated electrolyte disorders. Table 2 shows the association of these disorders with hospital mortality by univariate regression analysis with a comparison against the corresponding normal ranges. Hypernatremia, hyperchloremia, hyperkalemia, and metabolic acidosis combined with respiratory acidosis showed a significant association with hospital mortality (OR: 25.66, 18.86, 15.92, and 31.27, respectively, p<0.001). Similarly, the other disorders showed a significant association with mortality, and the respective ORs are shown in table 2.
Points-scoring system of electrolyte and acid-base disorders
In the derivation cohort, data pertaining to serum sodium, potassium, chlorine, CO2CP and AG levels were available for all patients. Data pertaining to serum calcium, phosphate, and magnesium levels were available for 57,904 patients, while results of arterial blood gas analysis were available for 12,438 patients. We respectively conducted multiple logistic regression analysis to analyze the associations of the above indexes with hospital mortality in three models, all of which were adjusted by age and gender.
The scores were assigned according to the regression coefficient and reserved as one significant digit. The theoretical maximum of the points-scoring system was 20.8 points. Severe hypernatremia (>150 mmol/L), AG >22.0 mmol/L, severe hyperphosphatemia (>1.8 mmol/L), severe hypermagnesemia (>1.3 mmol/L), pH <7.20, PaCO2 >60.0 mm Hg and base excess (BE) <−10 mmol/L were allocated over two points. However, we did not assign points for mild hypercalcemia (2.56–2.75 mmol/L, p=0.262) and hypomagnesemia (0.60–0.66 mmol/L, p=0.638) for no significant differences. All the regression coefficients, ORs, and scores are shown in table 3.
Significance of the risk score in predicting hospital mortality in the validation cohort
Then, we evaluated this risk score in the validation cohort. The median (IQR) of the risk score from the most severe status during hospitalization was 0.6 (0–1.3), and the range extended from 0 to 18.7 points. ROC curve analysis revealed that this risk score had a reasonable performance in predicting hospital mortality (figure 1A). Use of the optimal cut-off value of 2.0 to predict hospital mortality was associated with a sensitivity 65.4% and specificity 88.4% (area under the curve: 0.818 (95% CI 0.809 to 0.827)). According to this cut-off value of 2.0, the scoring system was divided into five intervals, including the interval with scores less than two points (Q1: 0 to 2) and four quartiles with scores >2 points (Q2: 2.1 to 2.5; Q3: 2.6 to 3.3; Q4: 3.4 to 4.5; and Q5: >4.5 points), and the proportions of these five parts were 84.5%, 3.9%, 4.1%, 3.6%, and 3.9%, respectively (table 4). The characteristics and outcomes of the validation cohort in the above groups are shown in table 4. The patients in the Q5 group with risk score >4.5 were older and had lower BMI, higher FBG levels and WBC counts, as well as lower albumin, hemoglobin, and SCr at admission (p<0.001). The hospital mortalities were 0.5%, 1.7%, 2.7%, 3.5%, and 11.7%, respectively, in the five groups, while the incidence rates of AKI also increased along with the increased risk scores. Then, Kaplan-Meier survival analysis showed significant differences in hospital survival between these five groups (p<0.001 between every group; figure 1B). We also divided the validation into nine intervals as shown in figure 1C,D. With the increase in the risk score, the frequency declined, while the hospital mortality increased significantly. The hospital mortality in patients with the risk score >12.0 points reached over 50%.
To identify the association between the risk score and hospital mortality in the validation cohort, we further carried out Cox proportional hazards models adjusted by age, gender, BMI, blood pressure, FBG, albumin, hemoglobin, WBC, and SCr at admission and the incidence of AKI (table 5). The risk score was an independent predictor of hospital mortality (HR 1.248, 95% CI 1.233 to 1.262, per 1-point increase, p<0.001) in model 1 adjusted by age and gender. The results of model 4 revealed that the HR was still 1.158 with 95% CI 1.133 to 1.183 after adjusting for all the above variables including the occurrence of AKI.
Significance of the risk score in dynamic monitoring of electrolyte and acid-base disorders in the validation cohort
The correction of hypernatremia (46.1%), hyperkalemia (63.6%), and metabolic acidosis presenting in combination with respiratory acidosis (37.1%) prior to discharge significantly reduced the mortality (14.3% vs 7.2%, 16.9% vs 6.0%, and 28.3% vs 5%, p<0.001). Therefore, we next assessed the association of the dynamic risk score with hospital mortality in the validation cohort. As shown in figure 2A, we divided the validation cohort into four parts according to the risk score at admission: 0–2, 2.1–4.0, 4.1–6.0 and >6.0. The dynamic risk scores of all patients and subgroups at admission, during hospitalization, and before discharge or death were significantly higher in the death group as compared with that in the survival group figure 2B–F.
The Kaplan-Meier survival analysis that was stratified according to the delta risk score (the risk score before discharge or death minus the risk score at admission; Q1: <−2; Q2: −2 to −0.1; Q3: 0 to 2; Q4: 2.1 to 4 and Q5: >4). The patients with increased risk score over 2.0 presented significantly lower hospital survival according to figure 3A. In addition, Kaplan-Meier survival analyses were also conducted to compare the differences in hospital survival of the subgroups with different levels of risk scores at admission (figure 3B–E). In the patients with risk score >2 at admission, an elevated risk score (delta score >2 points) or uncorrected risk score (delta score 0–2 points) significantly increased the risk of hospital death when compared with a decline in the risk score (delta score <0 points). In the patients with risk score <2 at admission, an elevated risk score (delta score >2 points) also significantly increased the risk of hospital death when compared with a stable risk score (delta score <2 points).
Discussion
In the present study, we prospectively enrolled a total of 322,046 patients, including 84,700 patients in the derivation cohort and 237, 346 in the validation cohort, in a large, tertiary hospital in East China. The prevalence, characteristics, and outcomes of electrolyte and acid-base disorders were analyzed. Most importantly, a point-scoring system with a theoretical maximum of 20.8 points was established to evaluate the severity of electrolyte and acid-base disorders and to predict hospital death. Thus far, this is the first evaluation system for the entire gamut of electrolyte and acid-base disorders. The optimal cut-off value of 2.0 was associated with a sensitivity of 65.4% and specificity of 88.4%, which indicates a reasonable performance of this risk score. Kaplan-Meier survival analyses showed that the maximum value of the risk score and that of the delta risk score was associated with hospital mortality. All these findings indicate the reliability of this novel and dynamic risk score for evaluation and monitoring of electrolyte and acid-base disorders, as well as to guide the corrective measures.
Herein, we demonstrated the overall epidemiology of electrolyte and acid-base disorders. Previously, only a small single-center study, which enrolled 112 patients from a general medical ward, had reported similar findings;8 in that study, 45% and 56% patients presented with electrolyte and acid-base disorders, respectively (58.5% and 62.9% in our study). In previous studies,5 21 hyponatremia was the most common electrolyte disorder among hospitalized patients. Similarly, we found that hyponatremia was the most common disorder (16.6%), followed by hypokalemia (13.7%) and hypochloremia (13.6%). The incidence of hypernatremia ranged from 0.3% to 2.6% among the general hospitalized population,22–24 which is similar to the 2.1% incidence reported in the present study. However, the prevalence of these disorders may vary according to various definitions. Some researchers adopted the upper limit of the local normal range, which varies worldwide; others may have used the definitions in textbooks or guidelines. Therefore, the incidence can vary more than 10-fold according to its definition.25 For instance, the definition of hyperkalemia in the published literature ranges from ≥5.1 mmol/L to ≥6.0 mmol/L.25 Martín-Pérez M et al26 found that the incidence of hyperkalemia in patients with heart failure decreased from 1.7% to 0.3% when the definition changed from 5.5 mmol/L to 6.0 mmol/L. Interpreting the results of studies and applying their results to clinical practice requires consideration of variation in definitions. Meanwhile, consensus definitions are needed for real improvement in diagnosis and therapy.
Furthermore, in our study, hypernatremia, hyperchloremia, hyperkalemia, and metabolic acidosis combined with respiratory acidosis showed a significant association with hospital mortality. Many studies have reported a significant association between hypernatremia and mortality.27 28 Hypernatremia and hyperosmolality may result in cellular dehydration, neurological impairment, myocardial depression, and even poor wound healing.1 The prevalence of hyperkalemia was reported from 1.3% to 3.3% of inpatients,29–31 and patients with hyperkalemia generally had poor outcomes. Hyperkalemia often occurs in patients with renal dysfunction. In the current study, estimated glumerular filtration rate (eGFR) <60 mL/min/1.73 m2 was found in 47.7% patients with hyperkalemia, which is similar to that reported in other studies (48%–74%).31 32 The combination of metabolic acidosis plus respiratory acidosis often occurs in critically ill patients with cardiac arrest, renal failure, severe congestive heart failure, as well as lactic acidosis with sedatives or chronic obstructive pulmonary disease.33 34 Few studies have assessed the association between mixed acid-base disorder and clinical outcomes. A study of 108 pediatric patients with traumatic brain injury showed that mixed metabolic acidosis plus respiratory acidosis were independent predictors of in-hospital mortality.35 Previous animal studies found that the combination of metabolic acidosis and respiratory acidosis may increase pulmonary vascular resistance and reduce stroke volume and myocardial contractility.36 37 To date, the analysis of mixed acid-base conditions, mainly using the Henderson-Hasselbalch equation and the Stewart approach, is still not easy for clinicians. In addition, interpreting acid-base disorders through the use of software, which began three decades ago, is another good option38 39 that can generate quick and accurate answers. Although it has not been put into clinical practice for many reasons, electronic interpreting and alerting of electrolyte and acid-base disorders should be developed for clinical use.
Indeed, acid-base and electrolyte disorders need to be monitored closely through electronic alerting. As discussed earlier, electrolyte and acid-base disorders can be induced by the development of underlying pathology, organ dysfunction, and improper treatment; therefore, this novel point-scoring system of electrolyte and acid-base disorders must be strongly associated with poor outcomes. After adjustment, the risk score was an independent indicator of hospital mortality (HR 1.158 with 95% CI 1.133 to 1.183, per 1-point increase). From the above results, we considered that the risk score of all hospitalized patients should be monitored, and a score below the optimal cut-off value of 2.0 may be a relatively safe range.
This points-scoring system aimed to predict the outcomes and to monitor these disorders dynamically, with an ultimate aim to guide clinical correction of these imbalances. In a cohort study of 7067 patients in 18 French intensive care units (ICUs), hypernatremia corrected by day 3 was independently associated with improved survival.40 Unfortunately, the correction rate in clinical practice has not been improved in recent years and less than 50% of the disorders were corrected properly.9 40 However, too quick or too slow a correction is harmful,41 and a decrease in the serum sodium level of 10 mmol/L in 24 hours is suggested.42 McMahon GM et al43 conducted a retrospective study among 39,705 ICU patients and found that with a potassium decrease of ≥1 mmol/L within 48 hours of ICU admission, the association between hyperkalemia and mortality was no longer significant. The correction of mixed acid-base disturbance was significantly lower than that of electrolyte disorders in our study, probably due to severe illness, or, more importantly, poor analysis and clinical management. Therefore, could this risk score be used to guide the correction of these disorders? We think this is possible. In the present study, Kaplan-Meier survival analyses were used to compare the differences in hospital survival according to the stratified delta risk score; the results indicated that an elevated risk score (delta score >2 points) after hospitalization obviously increased the risk of hospital death, whereas a decreased risk score (delta score < 0 points) reduced hospital mortality. This risk scoring system reminded us that electrolyte and acid-base disorders should be considered as a whole, and their correction should not be focused on merely one aspect.
Study limitations
However, several limitations should be noted. First, this points-scoring system was generated and tested through analyzing the same sample of hospitalized patients, and its performance in patients with various underlying diseases may be different. Second, the variables included in the scoring system may have potential associations (eg, pH, PaCO2, CO2CP, HCO3−, and BE), but we could not completely separate them. Third, although we aimed to develop a risk score to guide the correction of these disorders, its performance and significance in electronic alerting and monitoring need to be fully tested in future research.
Conclusions
Electrolyte and acid-base disorders are common among hospitalized patients and are highly associated with hospital mortality. This novel scoring system could be used to evaluate and to monitor the severity of electrolyte and acid-base disorders dynamically.
Footnotes
YW and JH contributed equally.
Contributors YW and JH contributed equally. Research idea and study design: YW, JH, JT, XD; data acquisition: YW, XG, XZ, XX, JL; data analysis/interpretation: JH, XZ, XX, JL, JT; statistical analysis: YW, JH; supervision or mentorship: XZ, XX, JL, JT. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved. XD takes responsibility that this study has been reported honestly, accurately, and transparently; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Funding This work was supported by the Shanghai Public Health Improvement Action Plan (Grant No.15GWZK0502), the National Natural Science Foundation of China (Grant No.81430015), and the Project of Science and Technology Commission of Shanghai Municipality (Grant number: 17140902300).
Competing interests None declared.
Patient consent Not required.
Ethics approval Ethics Committee of Zhongshan Hospital, Fudan University (B2018-175).
Provenance and peer review Not commissioned; externally peer reviewed.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, an indication of whether changes were made, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.