You are here

A qualitative and quantitative triangulation approach to validate the measurement of spiritual constructs

Meeting Preference

Online June Meeting

Only Submit to my Preferred Meeting

Study Background


Over the span of 10 years a faith-based NGO has tested over 140 unique faith and spirituality questions. This triangulation study was developed to identify which survey questions can be validated as predictive of where a person is in their spiritual journey.


Although capturing spiritual metrics can be quite abstract, the objective of the study is to isolate survey questions that are predictive of whether a person is categorized to be a mature  adherent to the faith by local church elders. Measuring spirituality is inherently complex, involving culture, context, theological leanings, and subjectivity. While it is worthwhile to explore these complexities further, in this study, the judgment of local church leaders was used as ‘true north’ on the maturity level of a respondent for a number of reasons. First, the perspective of local church members puts the process of categorization in the hands of those that would come from the same context as the respondents. Second, the method leverages on elder experience in  interviewing and discerning a person’s spirituality and understanding of the Bible. 


Triangulation Approach


The study uses the triangulation approach, where both quantitative (surveys) and qualitative (interviews) methods are applied. Ordinarily, surveys administered via interviews are the main stream of data for the organization. Instead of attempting to validate the questions through intra-survey methods, a mixed quantitative-qualitative method was chosen. 


In addition, the study utilizes back-to-back interviews to ensure robust qualitative results. Each participant goes through two interviews with two different interviewers on the same day. After their first interview, participants wait until they move on to their next interview with a different elder. The second interviews are video recorded and will go under review and analysis at a later time. Each interviewer is also tasked to categorize each person into: Skeptic, Seeker, New Believer,  or Growing Believer. 


Elders from a local church in the province of study were enlisted to assess where a respondent is on their spiritual journey, similar to baptism interviews. Given the elders’ breadth of knowledge and experience, they were provided free-reign on the questions they asked to study participants. These consecutive interviews shed some light on the complexities of spirituality, not to mention insights on the spirituality of the elders themselves. As such, the percentage of agreement versus disagreement in these elders’ assessments of the participants becomes an important facet of the study. 




The study uses the triangulation approach to validate 82 survey questions against the elders’ assessments of the participants’ spirituality. 200 participants were selected from a mix of communities. All 200 participants were surveyed by contracted enumerators. From these, only 150 participants were chosen for interviews - split among Catholics, Protestants, and a few atheists, agnostics or a religious - based on survey results. 


Once selected, quantitative responses are collected by means of enumerator-administered surveys while the elder assessments were gathered through back-to-back interviews. The survey results were converted into binary indices (correct answer = 1, incorrect = 0) and the spiritual categorizations given by the elders were tabulated. The elder scores for each individual were combined by taking the average, and in the analysis those with an average score under the Skeptic or  Seeker categories were treated as ‘Non-Believer’, and those that were New Believer or Growing Believer were treated as Believer. 


These two sets of data were analyzed together by means of correlation tests, regression analyses, and confusion matrices to identify key survey questions that best predict spirituality as defined by the church elders. Due to complexities in the data, sophisticated regression methods were explored. The following are the supervised machine learning techniques used during the methodology testing: regularization techniques, such as Lasso, Ridge, and Elastic Net regression, as well as Least Angle regression, which combines Lasso regression  and Stepwise regression. 


Preliminary Results


The models identified 9 survey questions that were good predictors for the interview results by at least 2 regression models:


  • I read or listen to the Bible on my own. 
  • What is your current religion or denomination?
  • Do you own a Bible? 
  • How active are you in this Church (Sunday Service)?
  • It doesn’t matter much what I believe so long as I am good. 
  • If I am good enough, God will cleanse me of my sins.
  • I show God's love to those around me through telling them about Jesus.
  • Eternal salvation is possible only through grace, not works. 
  • What type of church (Sunday Service) do you belong to?


The subsequent step after identifying the best predictors involves determining the most effective combination or combinations of questions that accurately identify individuals according to the categorizations given during the interviews. A confusion matrix approach was applied, allowing a specific set of questions to be identified that are linked with a positive and negative predictive value of interview categorization. These findings are currently being refined and will be ready to present at the annual meeting.

Abstract for Online Program Book (maximum 150 words)

Measuring spirituality is complex, involving culture, context, theology, and discernment. This study attempts to validate closed survey questions for predicting spirituality according to the judgment of local leaders. Employing a triangulation approach, it utilizes the assessment of local church leaders, gathered via participant interviews, to categorize individuals according to maturity, providing a benchmark to compare the survey results against. Advanced statistical models were then applied to identify key quantitative survey questions that were best correlated with the local leader categorizations. This methodology situates the ‘decision making’ determination of spiritual maturity into the hands of local voices, and seeks to connect this knowledge to commonly used survey instruments in the field of spiritual measurement.