54 Comments

Debunking the Jockers Study

I really appreciate a comment by Chris Spencer on my previous post Dueling Wordprint Studies.  In that post, I had discussed a controversial study completed by Stanford researchers Mathew Jockers, Daniela Witten, and Craig Criddle who concluded that 57% the Book of Mormon was authored by Sidney Rigdon and Solomon Spaulding.  (There was an interesting discussion at Mormon Matters as well.)  Part of the reason they had Rigdon and Spaulding as candidate authors was due to the Spaulding Theory.  Here’s a bit of background.

Ever since the Book of Mormon was published in 1830, critics have tried to show that it came forth as the result of fraud.  One of the earliest theories was the Spaulding Theory.  As the theory goes, Solomon Spaulding wrote an unpublished novel about a group of Romans from the time of Constantine that were blown off course from Britain to the Americas.  Somehow (never adequately explained) Sidney Rigdon obtained the manuscript, and then transferred it surreptitiously to Joseph Smith who added religious information.  Fawn Brodie put together an appendix in her book No Man Knows My History outlining problems with the theory.  (I wrote about this in a post called Debunking the Spaulding Theory.)  Most people think the theory has been debunked, though the theory still has some adherents, such as Dale Broadhurst who maintains a website in favor of the theory.

Wordprint studies try to determine the true author of text.  The idea of a wordprint is similar to a finger print.  Each person uses a certain set of words such as “a, but, and, the, etc” in a way that is unique.  By collecting information on word usage, a wordprint theoretically can identify an author.

In 2008, Mathew Jockers, Daniela Witten, and Craig Criddle of Stanford University created a stir when they produced a peer-reviewed article in Oxford’s journal titled Literary and Linguistic Computing.  The authors concluded that major portions of the Book of Mormon exhibited Sidney Rigdon and Solomon Spaulding’s writing style, thus creating a resurgence of interest in the Spaulding Theory.  Traditionally, wordprint studies have used a statistical technique known as the Delta Method.  Jockers, et al compared the Delta method to a new technique called Nearest Shrunken Centroid (NSC).  NSC has been used cancer studies, but this was the first time it has been used in wordprint studies.  The Jockers study found the NSC method to be much more reliable than the Delta method.  Many New Order Mormons and anti-Mormons were pleased with the study.  But there were some big questions about the method.

In January 2011 Bruce Schaalje, Paul Fields, and Matthew Roper of BYU, along with Gregory Snow of Intermountain Health Care released a study outlining problems with the Jockers study in the same Oxford journal of Literary and Lingustic Computing.  While acknowledging that NSC is a good method for wordprint studies, they detailed several problems with the Jockers study, noting a “naive application of NSC methodology” led to “misleading results.”  Jockers et al had used a closed set of 7 authors for their study.  Schaalje’s study showed that an open set of candidate authors “produced dramatically different results from a closed-set NSC analysis.”

The beginning of the Schaalje article discusses a bit of mathematical theory (I’ll spare you.)  Schaalje notes that this study has a foundation in theory, rather than emperical evidence like the Jocker study; therefore Schaalje’s study is a bit stronger.  Schaalje was able to reproduce Jocker’s results, and applied the same technique to another document: the Federalist Papers.  To demonstrate a problem with Jocker’s technique, they purposely excluded Alexander Hamilton from the list of candidate authors, and picked other authors to see which author the Jocker’s closed-set method would choose.  The candidate authors were Joseph Smith, early Sidney Rigdon, late Sidney Rigdon, Solomon Spaulding, Oliver Cowdery, and Parley P. Pratt.

Early or late Rigdon was falsely chosen as the author of 28 of the 51 Hamilton texts with inflated posterior probabilities ranging as high as 0.9999 (Fig. 2). Pratt was falsely chosen as the author of 12 of the papers, and Cowdery was falsely chosen as the author of the remaining 11 papers. These results dramatically demonstrate the danger of misapplying closed-set NSC.

Schaalje et al noted that Jocker’s et al should have used a “goodness of fit” test to verify how well their findings matched, and proposed a method to compute the goodness of fit.

An important extension to NSC classification is to allow an open set, i.e. the possibility that the test texts might not be authored by any of the candidate authors. We propose that this can be done by positing an unobserved author for each test text in addition to the observed candidates in the training data.  We propose an unobserved author with a distribution of literary features just barely consistent with the test text. Thus, as a straightforward extension of the NSC classification model, we suggest that posterior probabilities for the candidate authors be calculated as…

(Once again, I’ll spare the mathematical proof.)  They applied this goodness of fit test and said,

Applying this extended model to the Hamilton texts with Smith, early Rigdon, late Rigdon, Spalding, Cowdery, and Pratt as training authors, only 2 of the test texts were assigned to early or late Rigdon, while the remaining 49 were assigned to an unobserved author (obviously Hamilton)  (Fig. 5).

As a further test of the open-set NSC procedure, in addition to Rigdon, etc., we included Hamilton as a training author represented by the first 25 Hamilton papers. We classified the remaining 26 Hamilton papers as test texts. We first used the closed-set model. All 26 Hamilton test texts were correctly assigned to Hamilton; none was assigned to an unobserved author. The goodness-of-fit procedure (Fig. 4, right panel) indicated that the closed-set model was valid.  We then used the open-set model. All 26 Hamilton test texts were still correctly assigned to Hamilton. Hence, when the actual author was included in the training set, the allowance for an unobserved author as in Equation (10) did not appear to compromise the ability of open-set NSC to correctly attribute authorship.  It is important to note that the open-set NSC procedure does not indicate how many unobserved authors there are. All we know is that if an unobserved author is selected for a test text, one unobserved author is most probable as the author of that text. There could be as many unobserved authors as the number of test texts, or as few as one. A clustering procedure would provide some information as to the total number of unobserved authors.

In order for the NSC method to work, writing samples of test authors are needed.  Schaalje et al noted that the Jockers study has sample texts ranging from 114 to 17,979 words, with training texts ranging from 95 to 3752 words.  With such a wide disparity of sample texts, the BYU authors indicated that was another problem.

The measurement of 100 or more word frequencies on texts of less than 100 words is almost sure to produce unreliable measurements (Holmes and Kardos,2003). For the delta procedure, Burrows (2003, p. 21) found that ‘with texts of fewer than two thousand words in length… the test gradually becomes less effective’. Others have worked with texts of 1,000, 5,000, and 10,000 words (Larsen et al.,1980; Hilton, 1990; Holmes, 1992).

To test whether the size of the training text matters, the BYU authors used 8 Rigdon samples ranging in length from 100 to 5,000 words.

we recommend in general that the training data involve only texts of at least 1,000 words because feature-specific variances do not change greatly with text size beyond 1,000. Within limits, the problem of training text size variation can be dealt with simply by compositing shorter texts of known authorship to create training texts of at least 1,000–2,000 words.  Hoover (2004), in fact, found that combining several texts ‘helps to improve accuracy’ of authorship attribution.

After Sidney Rigdon left the church, he started his own church in Pennsylvania.  (I blogged about this group previously here and here.)  As noted in the 2nd link, Sidney Rigdon had many revelations between 1863 and 1873.  Schaalje used these revelations to see if any false positives could be attributed to another author using various size sample texts from Rigdon.

To illustrate the effects of both extensions (Equations 10 and 12) to the NSC method, we applied the closed-set NSC method and the two extensions to 95 ‘revelations’ attributed to Sidney and Phebe Rigdon between 1863 and 1873, decades after Rigdon had left the Mormon movement. The test texts ranged in size from 60 to 4,128 words. We used the Smith, Cowdery, Spalding, Pratt, and early Rigdon texts as the training data, and specified informative priors based on the fact that Smith, Cowdery, Spalding, and Pratt had all died long before 1863. The closed-set NSC model attributed the texts mainly to Rigdon and Smith (Fig. 7), the open-set NSC model attributed most of the texts to latent authors, and the fully expanded NSC model attributed the texts to Rigdon, Smith, and latent authors. The point here is that open-set NSC without adjustments for test texts sizes is inadequate if some of the test texts are very small.

In the Jockers study, they noted some false positives.  For example, Longfellow (one of the 2 control authors) was listed as an author of the Isaiah-Malachi chapters in the Book of Mormon.  Jockers noted the problem, but did not investigate further, feeling confident that Isaiah-Malachi was correct in 20 of 21 chapters.  Schaalje looked further into this “false positive” problem.

A disturbing feature of classification analysis when the set of test texts is large is that test texts on the stylistic fringe of the distribution for the true author can occur by chance, and may therefore ‘stray’ into the distribution of a nearby author.  This explains why 2 of the 51 Hamilton texts were assigned to Rigdon (Fig. 5), and might partially explain why 21 of the 95 late Rigdon texts classified strongly as writings of Smith (Fig. 7) even though Smith had died 20 years earlier. Historians who study this period would be hard-pressed to imagine any way that Rigdon could have retained otherwise unknown Smith texts.

The same problem was observed by Hoover (2004) with regard to the delta method. He noted (Hoover, 2004, p. 460) that for particular sets of authors and texts, ‘false attributions are a serious possibility’. Burrows (2002, p. 281) similarly cautioned that the ‘the system for distinguishing between insiders and outsiders is not foolproof’ because of its dependence on probabilities rather than absolutes.

This problem, which is exacerbated by heterogeneity in text sizes, is an example of the multiplicity or multiple comparisons problem in statistics (Benjamini and Hochberg 1995). One not completely satisfactory solution would be to composite all of the test texts into one or a few large texts, and then classify those texts. We combined the Sidney texts into two large texts, combined the joint Phebe–Sidney texts into one large text, and combined the Phebe texts into one text. Assigning realistic prior probabilities, the first Sidney text was classified to an unobserved author, the second Sidney text was assigned to Cowdery, the joint text was assigned to Cowdery, and the Phebe text was assigned to an unobserved author. These results indicate, at a minimum, that the authorship style of the late Rigdon texts was different from that of Rigdon’s earlier writings. This may be due to genre differences, the passage of time, or the interposition of editors. In any case, the cause of the difference is not germane to this study.

Schaalje doesn’t have a solution to the problem of false positives, but is continuing to study an idea to deal with the problem of unequally sized sequential texts.  Finally, Schaalje concluded with a very different conclusion from Jockers.

Using closed-set NSC, Jockers et al. (2008) attributed 37% of the chapters to Rigdon, 28% to Isaiah/Malachi, 20% to Spalding, 9% to Cowdery, 5% to Pratt, and 1% to Longfellow. In contrast, using open-set NSC, we conclude that 73% of the chapters cannot be reliably attributed to any of the candidate authors. We first note that Jockers et al. (2008) bolstered their NSC attributions by claiming close agreement between attribution results due to Burrows’ delta and those due to closed-set NSC.  That these stylistic measures would nominally agree well numerically is not surprising because Burrows’ delta stylistic distance is closely related to the quadratic delta stylistic distance (Argamon, 2008) upon which NSC is based.

However, there actually is strong disagreement between the closed-set NSC results and the delta results. This is because delta-z scores should not be taken seriously unless they are very small (i.e. very negative). Burrows (2003) found that a threshold of 1.9 separated most false positives from true attributions for a set of 17th-century poets. Jockers et al. (2008) failed to do this. In the Jockers et al. (2008) study, only 16 of the 239 chapters had delta-z values as small as 1.9 (Fig. 9). Ten of these 16 chapters were essentially verbatim quotations of Isaiah/Malachi, and all 10 were correctly attributed to Isaiah/Malachi. Four additional chapters were attributed to Isaiah/Malachi and the others to Rigdon and Spalding. The remaining 223 chapters had large delta-z values and were thus apparently false positive. Hence, the delta results of Jockers et al. (2008) actually say little more than what is already uncontroversial about Book of Mormon authorship: that some of the chapters are quotations of Isaiah and Malachi. The delta-z results do not, in fact, attribute sizeable percentages of the chapters to Rigdon, Spalding, or Cowdery.

I have to say that the BYU guys really thought through this problem well.  Jockers has plans for an updated study to include Joseph Smith, and other others.  Judging from the BYU study, I think the Stanford folks have some serious problems.  What are your thoughts?

Comment navigation

← Older Comments

54 comments on “Debunking the Jockers Study

  1. I am a geologist by training. In geology, we use seismic tomographic imaging to distinguish the various layers in the interior of the Earth. This technique is used to image the Earth’s core and large bodies of molten rock in the mantle. Results obtained by these techniques are often criticized on the basis that the usable results require massive amounts of data processing. Opponents often accuse researchers of “massaging” the data and the more massaging that takes place the more you can make the data look like anything you want.

    Having used a significant amount of statistics in my own studies it appears that we might have a similar situation with these wordprint studies. Large complex data sets can often be massaged to fit any interpretation you like. Furthermore, if I were to go out on a limb, I would say that both sets of authors (despite all attempts at objectivity) are coming at this issue with significant emotional baggage. I don’t think that Criddle (a former Mormon) and others will concede that the Book of Mormon is a historical document nor do I think the BYU chaps would concede the contrary. Because of this and the problems of data massaging I don’t think any headway will be made on either side of the aisle.

  2. Hi MH:

    I tend to agree with Chris Spencer. I think this is simply the next chapter in dueling word-print studies. I have to admit, however, that pretty much all of the math and some of the logic goes over my head. Matt Jockers has provided a response on another website (not sure if you want me to post a link, MH) in which he basically acknowledges Schaalje’s innovation in dealing with the closed vs open set issue, but stands by his previous conclusions. Among other things, he says (emphasis mine):

    I agree with Professor Schaalje that we must always worry about the possibility that the real author is not in the closed set, and we made that point in the conclusion of our paper. Professor Schaalje insists that even though we stated the point in clear language, that somehow we did not in fact state it, or perhaps we simply did not emphasize the point to his satisfaction. Again, I’d say this is a difference of opinion. From my perspective the nature of our experiment was perfectly clear.

    Professor Schaalje’s work in trying to deal with the closed vs. open set problem is innovative and will likely inform future work in the field. As a new approach to authorship problems, I think he and his team have likely made a valuable contribution. I do not, however, see Professor Schaalje’s paper as a refutation of our work, and perhaps this is because I read the results of our work differently than he does.

    The paper we wrote was designed to answer the question of who among the suspect candidates was the most likely. That’s it. In my opinion, Professor Schaalje’s paper takes aim at a fictionally constructed argument that we did not in fact make: hence my reference to the “straw man,” “playing fast and loose with our conclusions,” and “slight of hand.” I understand that Professor Schaalje and other readers may believe that our paper was about “proving” who wrote (or most likely wrote) the Book of Mormon. That is most certainly not what it was about for me, and this is why I do not believe that Professor Schaalje’s paper stands as a rebuttal of our work. Our work was designed to rank a closed set of candidates who had been suggested by other researchers as possible authors. From my point of view our results showed simply that one candidate in the set was more likely than another (for any given chapter).

    Later in the same post, Matt says this:

    I continue to stand by our work, and I will defend it for what it is. I cannot, however, defend what others may construe it to be. Frankly, I have no interest in the ongoing discussion of just what can or cannot be said–based on our work or Professor Schaalje’s–about who, ultimately, wrote the Book of Mormon. Given the present evidence, this is not a question that I (or probably anyone else) is in a position to answer, even with advanced statistics and computation! All that anyone can do is to further probe the evidence and hopefully reveal new information that will change the game.

    It’s interesting to note that Jockers himself has little interest in who wrote the Book of Mormon. That interest, rather, falls to his colleague, Craig Criddle, a prominent advocate of S/R.

    As I pointed out later in that same thread, for those of us who are interested in who authored the BOM, Schaalje’s results are only meaningful if Nephites really existed. If not, then there is either a flaw in Schaalje’s method or some 19th century author other than the candidate authors produced the text. I strongly suspect the former and I think–but can’t prove–that the flaw lies in the King James English emulation of the BOM vs the 19th century style of the candidates. This glaring disparity allows Schaalje to assign low probabilities to any potential 19th century author because none of those sample texts were intentional emulations of King James English. Frankly, I think that’s a huge factor.

    But again, what we are left with at this point is dueling word-print studies. I do not know whether there will be a formal response forthcoming from Jockers et al. I hope so. Informally, Jockers ran additional tests with Joseph Smith included and the results were different (as one would expect) but not radically so. Joseph Smith was selected as the author of a fairly small number of chapters, with the bulk still going to Rigdon and Spalding. It’s also interesting to note that on that same thread, LDS apologist Ben Mcguire has stated that (emphasis mine):

    The approach that Jockers and company took was, I think, quite valuable. It was something new – used some good tools, and so on. It had some flaws. This new paper does not destroy the Jockers methodology. It corrects it. It still uses most of the framework that the Jockers study erected. What it does is show that the flaws in the methodology resulted in bad conclusions in this particular test case. When the Jockers study has the actual author in the mix, it does very, very well (just as the open NSC method proposed in the paper under the same circumstances). When the author is not in the mix, it does very poorly – and there was no mechanism to determine whether or not the author was in the mix. What this paper does is introduce a mechanism to do just that – and in doing so allows the process to spit back a result that says that all of the potential authors are dissimilar enough to the test material to suggest that none of them are good candidates for authorship.

    If Ben is correct in that observation then Schaalje’s results don’t change anything–and that, I think is Matt’s point. Schaalje’s results do not prove that Jocker’s attributions are not valid. Rather they suggest that Jockers may have produced some false positives IF THE REAL AUTHOR IS NOT AMONG THE CANDIDATE AUTHORS. That, of course, is a big “if” that ultimately leads us back to our prior assumptions.

    One other quick observation, the Jocker’s attributions have been criticized by some (including Dale Broadhurst)for using chapters as opposed to smaller units. Best as I can tell, Schaalje does not correct this. The problem is easy to understand… if there were indeed multiple authors, how do we know one chapter was authored by only one author? The answer is: we don’t. In fact there could be multiple contributors to any given chapter and this not only complicates Jocker’s findings but Schaalje’s as well.

    At the risk of posting too much verbiage, I also feel compelled to comment a bit on your summation of S/R. Not at all suggesting that you are intentionally distorting anything, but some of the points you made are classic misrepresentations.

    Ever since the Book of Mormon was published in 1830, critics have tried to show that it came forth as the result of fraud. One of the earliest theories was the Spaulding Theory. As the theory goes, Solomon Spaulding wrote an unpublished novel about a group of Romans from the time of Constantine that were blown off course from Britain to the Americas.

    In actuality the theory postulates that Spalding wrote at least 2 manuscripts. There is no doubt he wrote at least one, because we still have it today. That is the Roman story to which you refer. The other is alleged to have existed by witnesses who knew Spalding well and claim to have been exposed to it on multiple occasions during Spalding’s lifetime, prior to his death in 1816. Then there are other witnesses who claim to have examined it after BOM publication in 1833. Beyond that, there are credible reasons to conclude it existed that would take too much space to go into here.

    It is this now non-extant manuscript that S/R theorists suggest was the basis (or at least a basis) for the BOM–not the Roman story. The Roman story is interesting, however, because it was written by Spalding and because it actually does have some similarities to the BOM as well as Joseph Smith’s 1838 discovery narrative.

    Somehow (never adequately explained) Sidney Rigdon obtained the manuscript, and then transferred it surreptitiously to Joseph Smith who added religious information.

    Actually there is more than one adequate explanation for how and when Rigdon likely obtained the now non-extant Spalding manuscript. In other words, Rigdon had more than one opportunity and, interestingly enough, it is this opportunity that he passionately denies in response to comments made by Spalding’s widow in 1839. He states that:

    It is only necessary to say, in relation to the whole story about Spalding’s writings being in the hands of Mr. Patterson, who was in Pittsburg, and who is said to have kept a printing office, and my saying that I was concerned in said office, &c. &c. is the most base of lies, without even the shadow of truth; there was no man by the name of Patterson during my residence at Pittsburgh who had a printing office; what might have been before I lived there I know not. Mr. Robert Patterson, I was told, had owned a printing office before I lived in that city, but had been unfortunate in business, and failed before my residence there. This Mr. Patterson, who was a Presbyterian preacher, I had a very slight acquaintance with during my residence in Pittsburgh. He was then acting under an agency, in the book and stationery business, and was the owner of no property of any kind, printing office, or any thing else, during the time I resided in the city.

    Here Rigdon throws up a red herring. It was Robert’s brother Joseph who had the publishing house. In any event, the explanation gets too involved for this post, but suffice it to say that documentary evidence discovered in the last decade shows Rigdon was lying about having ample opportunity to purloin a Spalding manuscript from the Patterson publishing house.

    Also, I know of no S/R advocate who thinks Joseph Smith was the one who added the bulk of the religious material. Rather the theory holds that Sidney Rigdon added it. By and large, the Jockers attributions support that as many areas focusing on religion are attributed to Rigdon.

    Fawn Brodie put together an appendix in her book No Man Knows My History outlining problems with the theory. (I wrote about this in a post called Debunking the Spaulding Theory.) Most people think the theory has been debunked, though the theory still has some adherents, such as Dale Broadhurst who maintains a website in favor of the theory.

    Every point Brodie makes can be refuted and her conclusions can easily be challenged. Also, it is not necessary to assume Sidney Rigdon, or for that matter Joseph Smith & Oliver Cowdery, were attempting to pawn fraudulent scripture to the credulous masses. On the contrary, I think Rigdon believed he had an authentic translation of an ancient manuscript that contained an actual history of the former inhabitants of the American continent and that he had been duly authorized by God to include additional authoritative revelations with it. I don’t think Rigdon would have considered it fraudulent at all. On the contrary, he would likely have seen it as God speaking religious and historical truths to the world through his servant Sidney.

    All the best.

  3. Roger, welcome back! I hope I was able to fix your quotes correctly with the “blockquote” tag.

    I emailed Craig Criddle to get a response. He emailed me back and said he can’t respond to every post, and briefly mentioned something of a “straw man” that I didn’t understand. I thought about posting his email here, but I wasn’t sure if it was good etiquette to post an email without asking if it was ok. Your comment was much more thorough anyway, so let me address a few things, especially the “straw man” point.

    I understand that Professor Schaalje and other readers may believe that our paper was about “proving” who wrote (or most likely wrote) the Book of Mormon. That is most certainly not what it was about for me…

    Well, if Schaalje and other readers misinterpreted the message, it wasn’t very clearly stated in the original paper, despite Jockers claim that

    “Professor Schaalje insists that even though we stated the point in clear language, that somehow we did not in fact state it, or perhaps we simply did not emphasize the point to his satisfaction. Again, I’d say this is a difference of opinion. From my perspective the nature of our experiment was perfectly clear”.

    Everyone I know that discusses this issue thinks the Jocker’s study “proves” the Book of Mormon was written by Rigdon/Spaulding. If so many people are misinterpreting the message, then it wasn’t written very clear, despite Jocker’s assertion of writing it in “clear language”.

    As I pointed out later in that same thread, for those of us who are interested in who authored the BOM, Schaalje’s results are only meaningful if Nephites really existed. If not, then there is either a flaw in Schaalje’s method or some 19th century author other than the candidate authors produced the text. I strongly suspect the former and I think–but can’t prove–that the flaw lies in the King James English emulation of the BOM vs the 19th century style of the candidates. This glaring disparity allows Schaalje to assign low probabilities to any potential 19th century author because none of those sample texts were intentional emulations of King James English. Frankly, I think that’s a huge factor.

    I frankly disagree with this characterization that Schaalje’s study is “only meaningful if Nephites really existed.” If the true author was someone else, such as Ethan Smith, then Schaalje’s study would confirm Ethan as the author. One need not conclude that Nephi, or Alma, or Mormon wrote it, though Schaalje leaves that possibility open, unlike Jockers. When we discussed this earlier, I wondered why Ethan Smith wasn’t included as a possible author in the Jocker’s study. Schaalje’s open set model is much more robust than Jocker’s closed-set model, and I just don’t see how Jocker’s can stand by his former conclusions when the goodness of fit test shows that Rigdon/Spaulding don’t fit.

    If Jockers wants to claim that Rigdon/Spaulding is the best fit of those specific 7 authors–fine. But people will inevitably jump to the conclusion that Jockers “proves” S/R were the authors, and I just don’t understand how Jockers can claim that is the fault of the reader. His attempts to “clarify” the message are not satisfactory, IMO. I think the Jockers study is fatally flawed, and when viewed against the open-set method, it is just plain inferior.

    As for my “mischaracterizing” the Spaulding theory, I was trying to state it simply. I wanted to focus on the Schaalje study, not the Spaulding theory, and it is hard to put all the caveats in a short-hand version. I linked to our previous discussion and Dale Broadhurst’s website for people to review for further detail.

    But suffice it to say, I am not persuaded by the arguments that Rigdon surreptitiously transferred the document to Joseph Smith, or this idea that if Rigdon had obtained the Spaulding manuscript, that “Rigdon believed he had an authentic translation of an ancient manuscript that contained an actual history of the former inhabitants of the American continent and that he had been duly authorized by God to include additional authoritative revelations with it.” The logic just doesn’t flow to that conclusion, IMO.

    Rigdon started his own church in Pennsylvania following the death of Joseph Smith. He was trained as a Baptist preacher, and did not need Joseph Smith to start a church. (He was already on that track of starting his own church–or at least pastor of a congregation–in the late 1820’s.) He could have easily claimed to have translated the Book of Mormon at the same time that Oliver tried to translate, or he could have claimed to have received the Spaulding manuscript by angelic visitation or revelation, or some other holy endeavor on his own and bypassed Smith completely. The whole conspiracy aspect just makes no logical sense to me.

    The Spaulding theory went for over 30 years (roughly 1830-1860) on a missing manuscript. When “Manuscript Found” was discovered and deemed unworthy, now proponents have to link to a 2nd missing manuscript. After Spaulding’s manuscript was found in the 1860’s (?), I find no compelling reason to believe he authored a second non-extant document. “Manuscript Found”, the story of Fabius, is not compelling to me. The 2nd manuscript argument just isn’t at all convincing to me. The story of Fabius is laughable, and I don’t think Spaulding had the ability to write something better, especially along the lines of the Book of Mormon.

  4. MH:

    Thanks for the welcome back and for fixing my post! I guess I will simply need to use quotation marks or the double arrow >> in the future. I also want to commend you for tolerating and even soliciting opposing points of view for your blog. I appreciate your willingness to allow open dialogue. Very commendable.

    I’m glad you did not post Craig’s email without his approval because the very reason Matt responded on the thread I was referring to was because a person he had sent a private email to posted it online without his prior approval. He felt betrayed and although I certainly appreciated what Matt had to say, I wish it wouldn’t have come about the way it did. With Craig being a close associate of Matt’s, I am sure he is a bit leary now as to what he writes in a personal email.

    Since you said it is okay to post links, here is the thread I was referring to, I am sure you will find it interesting:

    http://www.mormondiscussions.com/phpBB3/viewtopic.php?f=3&t=16575&st=0&sk=t&sd=a&sid=be46754049801d4f4718698ad392a5f5

    >>Well, if Schaalje and other readers misinterpreted the message, it wasn’t very clearly stated in the original paper, despite Jockers claim that
    “Professor Schaalje insists that even though we stated the point in clear language, that somehow we did not in fact state it, or perhaps we simply did not emphasize the point to his satisfaction. Again, I’d say this is a difference of opinion. From my perspective the nature of our experiment was perfectly clear”.

    Everyone I know that discusses this issue thinks the Jocker’s study “proves” the Book of Mormon was written by Rigdon/Spaulding. If so many people are misinterpreting the message, then it wasn’t written very clear, despite Jocker’s assertion of writing it in “clear language”.

    I’m not sure who you are having discussions with, MH, but I don’t know of anyone who claims Jockers’ results prove anything. My own opinion is that the Jockers’ results support the S/R thesis–a thesis, by the way, that I think already has plenty of support with or without the Jockers’ results.

    >>I frankly disagree with this characterization that Schaalje’s study is “only meaningful if Nephites really existed.” If the true author was someone else, such as Ethan Smith, then Schaalje’s study would confirm Ethan as the author. One need not conclude that Nephi, or Alma, or Mormon wrote it, though Schaalje leaves that possibility open, unlike Jockers. When we discussed this earlier, I wondered why Ethan Smith wasn’t included as a possible author in the Jocker’s study. Schaalje’s open set model is much more robust than Jocker’s closed-set model, and I just don’t see how Jocker’s can stand by his former conclusions when the goodness of fit test shows that Rigdon/Spaulding don’t fit.

    The debate reaches a certain level where it becomes a battle of the experts. I am not a statistician or a scientist or even a linguist. I can only go off what the experts say, and at this point we have dueling experts with a lot of subjectivity. As I understand it, Schaalje’s point is that if the real author is not in the mix, the results are going to show false positives. If I understand what Matt is saying, he agrees with that and also seems to be saying he never claimed anything more. I can live with that. You may be correct, in that it is POSSIBLE that even if there were no actual Nephites, the real BOM author may not have been in the candidate list. What Matt is saying, and I concur, is that their study examined the most likely suspects and of those determined which best matched any given BOM chapter. Even Ben Mcguire acknowledges that if the real author is in the mix, Jockers’ methodology works “very, very well.” Again, as someone who has already accepted the S/R thesis for other reasons, I am quite happy with that.

    With regard to Ethan Smith, I say, sure, run more tests with Ethan included and see what happens. Dale has also suggested testing Lucy Mack Smith and others like John Whitmer. Sure, why not? I think the issue is that it takes a lot of time, thought and effort (and possibly money?) to submit a study like that to a peer-reviewed journal, so any response–if one is even in the making–will take a while.

    >>If Jockers wants to claim that Rigdon/Spaulding is the best fit of those specific 7 authors–fine. But people will inevitably jump to the conclusion that Jockers “proves” S/R were the authors, and I just don’t understand how Jockers can claim that is the fault of the reader. His attempts to “clarify” the message are not satisfactory, IMO. I think the Jockers study is fatally flawed, and when viewed against the open-set method, it is just plain inferior.

    Well I can’t speak for Matt, but he has flatly stated that he really has no dog in the fight. Frankly, he doesn’t care who wrote the BOM. At least that’s what I get from his remarks. The person who cares is Craig Criddle, who, like me, has already concluded that S/R best explains the BOM. Jockers results–which anyone could duplicate–support that. They don’t prove it. They simply support it.

    Best as I can tell, Matt is simply saying, look, we took what appears to be the most likely 19th century subjects, tested them and here’s the results. Take it for what you want. Nothing more, nothing less. As someone who thinks SR is the best explanation I see the results as supportive of that framework. And I don’t think Schaalje has changed anything in that regard.

    >>As for my “mischaracterizing” the Spaulding theory, I was trying to state it simply. I wanted to focus on the Schaalje study, not the Spaulding theory, and it is hard to put all the caveats in a short-hand version.

    I agree and hopefully I wasn’t too harsh. I know you weren’t trying to mischaracterize anything. It’s just that as someone who’s looked into what S/R claims, I am sensitive to inaccuracies that creep into descriptions of what S/R is allegedly all about. You mentioned Brodie, and she really messes up. I fully agree that S/R is complex and is very difficult to accurately summarize–especially so when you don’t believe it!

    >>I linked to our previous discussion and Dale Broadhurst’s website for people to review for further detail.
    But suffice it to say, I am not persuaded by the arguments that Rigdon surreptitiously transferred the document to Joseph Smith… The logic just doesn’t flow to that conclusion, IMO.

    The points you raise here are legitimate challenges to S/R. It’s difficult to adequately respond in a blog because to do the matter justice I would really need a lot more space (and time!) but let me give it shot and see if I can at least summarize in a way that makes sense. Obviously we are still going to disagree, but at least maybe you can see logic that actually flows.

    I think Rigdon believed he had an ancient manuscript. Here’s why. The opening of the EXISTING Spalding manuscript starts out with a description of how the author, Solomon Spalding, discovers his own ancient manuscript while walking near his home. Sound familiar? The thing is, Spalding wrote this sometime around 1812–well before Joseph Smith produced his own account in 1838 with some remarkable parallels to Spalding’s. Spalding’s EXTANT discovery narrative, at the beginning of MSCC (Manuscript Story Conneaut Creek) then goes on to suggest that Spalding himself translated this ancient manuscript from Latin into English. Sound familiar?

    So, here’s what I think happened. Spalding likely included a very similar account on his now non-extant manuscript–the one that would later be purloined by Sidney Rigdon. I think Rigdon simply believed what Spalding had written as fiction. Spalding himself said that in time his fiction would be accepted as actual history. He wrote it in a way that was meant to be taken as a valid account of the former inhabitants of America–and yet, everyone who knew him knew he was writing a fictional account.

    When Rigdon obtains the manuscript, he has no idea what Spalding’s intentions were. Spalding is dead at that point and can’t be consulted. So I think Rigdon simply takes Spalding’s fiction as a true account. But Rigdon was also delusional. He had a severe accident as a youngster, falling from a horse while his boot caught in the stirrup. He was dragged violently and severely injured his head. People who knew him well all agreed this affected his behavior and thinking. He truly believed he could receive revelations from God and in fact he produced many alleged revelations, just like Joseph Smith–in fact some were produced together.

    So whether you agree with my conclusions or not, perhaps you can at least understand the logic behind my conclusions. Rigdon finds a manuscript from a dead guy that claims to be a translation of an ancient text giving the history of the former inhabitants of America. It almost seems that God himself is placing this precious manuscript in his care now that the translator is dead. And why? Well because God obviously has truth to reveal through it and he has chosen his servant Sidney to complete the task by adding additional revelation and then presenting it to the world.

    >>The whole conspiracy aspect just makes no logical sense to me.

    Again, you make valid points. Here’s what I have concluded. Rigdon in fact needed Joseph Smith. He was too close to Spalding’s hometown to make it work. He knew that people would realize where this whole thing came from if he attempted to reveal it to the world (in fact that happened anyway! People eventually began to put the puzzle together anyway!)–even though, at the same time he also knew God was really using him to bring forth this great, new revelation. What to do? God must have some other anointed one to help bring forth this new revelation. But who?

    It is interesting to note that Spalding’s EXTANT manuscript speaks of seer stones. I think the Gazelem story in the BOM, or something like it, appeared in the Spalding manuscript Rigdon had purloined. And if Rigdon believes what Spalding had written, he would have been very impressed with a young man who could use a seer stone. This explains why Rigdon would put so much confidence in young Joseph Smith. He saw Smith as God’s anointed who was divinely put into his life at this key time in order to help bring forth the revelation in the Latter days that would initiate the gathering of Isreal and usher in the glorious millenium.

    >>The Spaulding theory went for over 30 years (roughly 1830-1860) on a missing manuscript. When “Manuscript Found” was discovered and deemed unworthy, now proponents have to link to a 2nd missing manuscript.

    Not quite correct. The Roman story was shown to at least one Conneaut witness (Aron Wright) in 1833 who flatly stated that it WAS NOT the manuscript he and the other witnesses were referring to in their testimonies. AT THAT TIME he said Spalding had written more than one manuscript and in fact, if you check the 1833 statements you will see that others of the Conneaut witnesses stated in 1833 (before the Roman story was even discovered)that Spalding had written more than one manuscript. Even Howe mentions the Roman story in his 1834 book and says it is obviously not the right one. The Roman story was simply RE-discovered in 1884, and a big (unnecessary) hoopla erupted, as sponsored by the RLDS church.

    >>After Spaulding’s manuscript was found in the 1860′s (?), I find no compelling reason to believe he authored a second non-extant document. “Manuscript Found”,

    Well you are not alone in that. In fact the idea that Spalding only wrote one manuscript is the primary basis S/R critics use to simply dismiss the rest of the S/R claims. But in my opinion there are several quite compelling reasons to believe he did. For one thing, MSCC is anything but publication-ready. The Spalding manuscript submitted to the Pattersons was alleged to have only lacked a Title and Introduction. In addition, one of the witnesses claimed Spalding wrote it on Foolscap paper–and that statement was made before MSCC was discovered. MSCC is not written on Foolscap. In addition several of the witnesses claim Spalding grossly over-used the phrase “and it came to pass” which does not occur in MSCC. Beyond that, otherwise credible witnesses with apparently no axe to grind–all of whom knew Spalding well–tell us he wrote more than one manuscript and one of them flatly denies MSCC is the one he was referring to.

    Again, you may not agree, but at least maybe you can see the logic behind the conclusion.

    >>the story of Fabius, is not compelling to me. The 2nd manuscript argument just isn’t at all convincing to me. The story of Fabius is laughable, and I don’t think Spaulding had the ability to write something better, especially along the lines of the Book of Mormon.

    I agree, the Fabius story is not the greatest literary prose, but then, parts of the BOM aren’t either and, interestingly enough, it is the parts Jockers attributes to Spalding that most resemble Spalding’s extant writing. Dale predicted this before desk-top computers even existed. How would you explain that, MH? Coincidence? Good guessing?

    All the best!

  5. Roger, you and (my other commenter) Tara write the longest comments of anyone I know! My that’s a lot to wade through. 🙂

    You’re definitely very meticulous in your word choice (and your advocacy of your position.) Your whole point over “proves/supports” seems to be quite a semantic argument, and splitting hairs. Fine, go ahead and substitute the word “support” every time I wrote “prove”, and I still stand by it all. Yet you’re not quite so meticulous on the Jockers/Schaalje method, as seen when you say,

    Even Ben Mcguire acknowledges that if the real author is in the mix, Jockers’ methodology works “very, very well.”

    Well, that is a big IF! There are foundational assumptions to both studies here, and that is the sample set of the test authors. The BYU guys leave open the possibility that the author is not in the set of test authors. Jockers doesn’t address this issue, and it appears to be highly influenced by Criddle’s bias in selecting the author set. If you want to prove the BoM is a piece of 19th century literature, Schaalje’s method leaves you an opening there, but Jocker’s will lead to far too many false positives. As I said in the previous discussion we had, if we picked you, me, Tara, FireTag, Craig Criddle, Matthew Jockers, and Bruce Schaalje, the Jockers method will pick 1 of us as the writer of the Book of Mormon, yet clearly none of us were even alive in 1830. The Schaalje method gives the option of “none of the above”. That’s a critical difference between the studies. Jockers depends far too heavily on getting the test authors correct, while Schaalje’s method doesn’t care if the correct author is in the mix because of the “none of the above” option. That’s a serious flaw of Jockers, and I don’t think this becomes a “battle of the experts” as you mentioned above. Schaalje’s method is clearly superior, and will allow Ethan Smith to be chosen as the real author if the goodness of fit test identifies him as the correct author. It also allows Alma to be an author (or not be an author) if we could get a handwriting sample from him! 😉

    I’m not here to debate the Spaulding theory again, but I will address your points if you really want me to. I think it would have been much easier for Rigdon to have been a translator of the Book of Mormon if he really was involved in the origins of the book than all the conspiracy one is required to believe to accept the Spaulding theory. That’s far too much baggage to believe. I’m sure you’re not surprised that I’d rather believe an angel appeared to Joseph than all the contortions to connect Sidney to Joseph prior to 1830.

    I think we both believe that God appeared to Moses. I am curious if you are aware of an Oxford publication claiming that Moses copied the 10 Commandments from Hammurabi? Is that a more likely explanation than God’s appearance to Moses?

  6. MH:

    Sorry my posts are so long. You can divide them up into chapters if it helps! LOL.

    >>Well, that is a big IF.

    Yes and no. Again, it goes back to one’s prior assumptions. Believers in the BOM will obviously think the real author(s) are not among the test authors because they died long ago. Those who reject that idea look at the candidate set (when it includes Joseph Smith) and ask: who is more likely to have contributed text to the BOM than these guys? You suggest Ethan Smith, and, again, I say sure, let’s test Ethan, but I doubt that Ethan contributed text to the BOM–willingly, at least. A cursory reading of Ethan’s style suggests some differences to most of the BOM–which is a point I’ll come back to in a moment. But as I understand the whole non-contextual word-print thing the idea is that we subconsciously use non-contextual words like “and” or “of” in unique patterns that only a computer can pick up. That’s why it is very significant (IMO) that the computer seems to agree with Dale’s observations that were made decades ago.

    Coming back to the other point, as you well know, the BOM is not a homogeneous piece of literature. It does not claim to be homogeneous and even claims to have more than one final abridger. If BOM critics who claim Joseph Smith is the only one responsible for the content are correct, then we should see Joseph’s fingerprint throughout the BOM, both contextually and non. But that doesn’t happen. So at least you and I can agree that there was very likely more than one person contributing content to the text. Sandra Tanner, for example, interprets that phenomenon as evidence of plagiarism. Well possibly so but one is hard pressed to find obvious examples of direct plagiarism other than that of the KJVB. And if direct plagiarism is not the explanation, then how could Joseph Smith adjust his patter of non-contextual words when only borrowing general concepts? Brigham Roberts, for example, suggests that View of the Hebrews may have been used as a framework for the BOM, and that’s possible, but I can find no clear-cut examples of direct plagiarism from VOTH to BOM. Rather, what I see is a lot of similar general ideas. Well how would this translate into a non-contextual word pattern a computer can pick up? I don’t think it could.

    But the parallels become more pronounced when we compare the BOM to Spalding’s extant manuscript. Coinciding with that is the Jockers data that attributes certain chapters to Spalding. The problem with that, however, is that no one claims that manuscript was the basis for the BOM. Rather it’s author is alleged to have written another manuscript that is claimed to have been the basis for the BOM. So, it gets complicated!

    But the fact that the BOM is not homogeneous is significant in that we should be able to measure the book’s internal claims with word-print authorship results. We should see a difference between the writing styles of the various ancient authors and redactors and we should see those differences lining up with what the book claims about itself. What Jockers’ results show is consistency with the prior claims of S/R, rather than those of LDS apologists–namely that by and large Rigdon is attributed to chapters we would expect him to have produced given the content of those chapters, Smith is attributed to chapters with content we would expect from him and Spalding as well. That is what is so significant about the Jocker’s data. It doesn’t prove anything, but if the BOM was indeed authored by ancient Nephites, the consistency of attribution with what S/R theorists predicted based on parameters different from non-contextual word patterns is a phenomenon that needs to be explained.

    >>As I said in the previous discussion we had, if we picked you, me, Tara, FireTag, Craig Criddle, Matthew Jockers, and Bruce Schaalje, the Jockers method will pick 1 of us as the writer of the Book of Mormon, yet clearly none of us were even alive in 1830.

    Possibly so, but what do you think the results would be if we included Rigdon, Spalding, Smith and Cowdery?

    >> The Schaalje method gives the option of “none of the above”. That’s a critical difference between the studies.

    Again, possibly so. More time is needed for the input of other people much smarter than me to determine whether Schaalje’s “none of the above” is truly effective. I will say that the sources I’ve been hearing from are skeptical. I don’t entirely follow everything, but again, the jist seems to be that Schaalje is apparently able to assign low probabilities to the 19th century authors (indeed ANY 19th century author) for some reason. I suspect that reason is the King James language of the BOM, but at this point I can’t prove it. Dale Broadhurst, I am confident, suspects the same thing. Schaalje produced a PCA chart and published it on MADB but he was reluctant to release which BOM chapters corresponded with the dots on his PCA chart when Dale asked for that information. Through a lot of determination, Dale was able to run his own tests and figured out which points were which. The results were very significant in terms of where certain BOM chapters clustered. It is a very important element to this discussion, but one that is not being discussed much, at least by Mormon apologists who would rather point to Schaalje’s conclusions than examine the factors involved in getting there.

    I suggest you invite Dale to share or at least describe his findings here.

    >>I think it would have been much easier for Rigdon to have been a translator of the Book of Mormon if he really was involved in the origins of the book than all the conspiracy one is required to believe to accept the Spaulding theory. That’s far too much baggage to believe. I’m sure you’re not surprised that I’d rather believe an angel appeared to Joseph than all the contortions to connect Sidney to Joseph prior to 1830.

    Well that’s fine, MH. You don’t have to drop your beliefs because I disagree with you. All I am saying is what may look on the surface as something totally illogical to you (and others) may only appear that way because A.) it IS complex and B.) it gets distorted by repeated mischaracterizations.

    One thing anyone who is honestly looking into S/R needs to keep in mind is that Sidney Rigdon was not a rational person. He suffered from delusions. Even you would have to admit that–I think–if I pressed you on it, because he claimed to receive revelations that you would flat out reject, but that he claimed came directly from God. Now if you can agree with me on that much, then you might at least be able to see that the expectations you have as a rational, 21st century, intelligent person are not necessarily going to be shared by Sidney Rigdon. He is not bound to act like you think a rational person would act.

    >>I think we both believe that God appeared to Moses. I am curious if you are aware of an Oxford publication claiming that Moses copied the 10 Commandments from Hammurabi? Is that a more likely explanation than God’s appearance to Moses?

    I don’t know. Anything is possible. But the difference is that Moses was very likely a real human being who actually lived in the real ancient world. In any case, the ancient Isrealite migration into Palestine is not challenged so far as I am aware. By contrast, there is zero evidence that Nephi or any of his descendants ever roamed the ancient American continent.

    All the best, my friend!

  7. Roger,

    Can you explain how Sidney Rigdon came out as author of the Federalist Papers using Jocker’s method? Is that a red flag to you?

    I’ve invited you, Dale Broadhurst, John Hamer, and Craig Criddle to join the discussion. Only you and Craig have responded so far; Craig said he was too busy to participate.

    But the difference is that Moses was very likely a real human being who actually lived in the real ancient world. In any case, the ancient Isrealite migration into Palestine is not challenged so far as I am aware. By contrast, there is zero evidence that Nephi or any of his descendants ever roamed the ancient American continent.

    OH CONTRAIRE, my friend! The Exodus is another of my favorite topics, and I blogged about it when I discussed Questions about the Exodus”. I don’t mean to sidetrack the discussion, but the similarities between the Exodus and the Book of Mormon are rather interesting, because (to use your phraseology) the Exodus has “zero evidence that [Israeliste] descendants ever roamed the [Sinai Peninsula].” If you may indulge me for a few moments, let me discuss some quotes from my post:

    **During Passover celebrations in 2001, Rabbi David Wolpe created international headlines in Israel by proclaiming to his Jewish congregation in Los Angeles, “the way the Bible describes the Exodus is not the way it happened, if it happened at all.”

    **Prof Philip Davies, University of Sheffield, “When it comes to the Exodus, we have no evidence that it happened, and a good deal of evidence that it didn’t. They made it up.”

    **Professor Eric Cline, George Washington University, “We do not have a single shred of evidence to date. There is nothing archeologically to attest to anything from the biblical story: no plagues, no parting of the Red Sea, no manna from heaven, no wandering for 40 years.”

    **Dr. Kathlyn M. Cooney, Egyptologist, Stanford University,, “the most likely reason that we’re not finding any evidence for the Exodus in Egypt is that it didn’t happen the way that the Bible said it did, or that it didn’t happen at all.”

    Now, for the record, I am with you that I believe Moses saw God, and there was an Exodus, and my post describes many ways the plagues could have happened, just as you have tried to describe how Rigdon could have received a Spaulding manuscript. My point in bringing this up is that if you’re going to base your reasons entirely on science, you’re going to have some problems. For someone who has studied the Spaulding issue as deeply as you have, I don’t understand how you can seemingly ignore these same issues with the Bible. I’d be very curious to find out if Dale Broadhurst believes in the Exodus (since you’re appealing to him as an authority), and I suspect Craig Criddle would believe the Exodus never happened too. I know I keep hitting this over and over with you, but I’m trying to point out some biases that seem to be complete blind spots for you.

    Anyway, I know that this post is not about the Exodus. However, I really encourage you to read up on that. I think there are far too many questions there that you are ignoring. I’m trying to be well-rounded here and look at skeptical points of view on not only Mormonism, but Judaism and Christianity as well. You seem to have a one-track mind on this Spaulding Theory, and highly discount the weaknesses in the conspiracy theory, as well as the Jocker’s study. I understand we both have our biases and come at this from critical/advocacy points of view. I’ve tried to give credit to Jockers where I felt it was due (his method was novel and an interesting addition to the field of wordprint studies), but the red flags are just too big to brush off for me. I don’t see big problems with Schaalje’s methodology at this time. If Dale wants to come here and discuss these probability problems you mention, I’m all ears.

    As I quoted before, the Schaalje study shows “Four additional chapters were attributed to Isaiah/Malachi and the others [2 chapters] to Rigdon and Spalding.” That’s a far cry from the 136 (roughly) chapters attributed to them by Jockers. Let’s not forget that Jockers had Longfellow as the author of one of the Isaiah chapters, yet the Schaalje method got that right. Between this an the Federalist Papers exercise, I don’t think that bodes well for Jockers. It appears to me that the Schaalje method is a significant improvement over the Jockers method, especially when it is clearly demonstrated as flawed in the Federalist Papers exercise. Jockers has done no such reliability testing. Yet the Schaalje method will still allow a skeptic to find the real author if the author is included as a test author. At this time, the weight of evidence argues against Rigdon/Spaulding as legitimate authors of the Book of Mormon, IMO.

  8. MH:

    Okay I’m going to respond, but then I really need to pack it in for the night.

    >>Can you explain how Sidney Rigdon came out as author of the Federalist Papers using Jocker’s method?

    No. Nor would my explanation count for much anyway. I will wait to hear what Craig and the others have to say.

    >>OH CONTRAIRE, my friend! The Exodus is another of my favorite topics, and I blogged about it when I discussed Questions about the Exodus”.

    What I said was: “the ancient Isrealite migration into Palestine is not challenged so far as I am aware.”

    Now it may be that my knowledge is lacking on this, but I did not suggest that the Biblical account is not challenged. I said the Isrealite migration into Palestine. Somehow, Isrealites got into Palestine and I don’t think anyone disputes that David & Solomon were actually kings there. In fact, evidence for their kingdoms has been found. But, yes, I agree, that is a sidetrack to the issue you raised on this thread.

    >>Now, for the record, I am with you that I believe Moses saw God, and there was an Exodus, and my post describes many ways the plagues could have happened, just as you have tried to describe how Rigdon could have received a Spaulding manuscript. My point in bringing this up is that if you’re going to base your reasons entirely on science, you’re going to have some problems.

    But I don’t. What I do, do, however is attempt to avoid scenarios where what I am accepting on faith is actually conflicting with what is known about the real world. Of course I am only human and can’t always achieve that objective, but it’s a goal to shoot for nonetheless.

    >>For someone who has studied the Spaulding issue as deeply as you have, I don’t understand how you can seemingly ignore these same issues with the Bible.

    Yes, this seems to be a recurring theme whenever I have discussions with LDS that challenge their beliefs. Regardless of what the specific topic actually is, they become very concerned about my biases–which I never deny having. I’ve come to accept it as a common phenomenon among LDS and simply choose to be flattered that they take such a keen interest in what they perceive to be the double standards I allegedly hold.

    >>I’d be very curious to find out if Dale Broadhurst believes in the Exodus (since you’re appealing to him as an authority),

    Good question. If I were to venture a guess I would think he would agree that Hebrews migrated into Palestine from the south, but that’s about as far as he would be willing to go. I could be totally off on that, though.

    >>and I suspect Craig Criddle would believe the Exodus never happened too.

    So far as I am aware, Craig is an atheist so, no, I doubt that he would be inclined to accept any of the Exodus account. Even he might agree that Hebrews migrated north into Palestine, though.

    So do I get any kudos for respecting both Dale and Craig’s opinions on S/R even though we may disagree on other things, or will that simply be added to the other double standards I allegedly hold?

    >>I know I keep hitting this over and over with you, but I’m trying to point out some biases that seem to be complete blind spots for you.

    MH, in ALL candor, when have I EVER denied having biases? Please pull the quote because I can’t remember making it. But the thing you need to keep in mind is that you have biases too. Can we agree on that? The whole bias thing is just a distraction, my friend. Virtually everyone on planet earth is biased.

    >>Anyway, I know that this post is not about the Exodus. However, I really encourage you to read up on that.

    When I have studied up enough and found what I consider to be enough answers as to where the BOM came from, my next goal is to take a closer look at legitimate criticisms of the Bible and/or the concept of “God” see how it/He fares. I’m slow. But as the wise old Buddhist monk says in “High Road to China” …the ox is slow, but the earth is patient.

    >>I think there are far too many questions there that you are ignoring. I’m trying to be well-rounded here and look at skeptical points of view on not only Mormonism, but Judaism and Christianity as well.

    Well for the time being that is out of my scope. I only have so much time. You asked me here to give my opinion on the Jockers/Schaalje studies and that’s what I’ve done, for whatever it’s worth.

    >>You seem to have a one-track mind on this Spaulding Theory, and highly discount the weaknesses in the conspiracy theory, as well as the Jocker’s study. I understand we both have our biases and come at this from critical/advocacy points of view. I’ve tried to give credit to Jockers where I felt it was due (his method was novel and an interesting addition to the field of wordprint studies), but the red flags are just too big to brush off for me. I don’t see big problems with Schaalje’s methodology at this time.

    Well that’s fine then. We simply disagree and, not surprisingly, we each think the study that supports our previous ideas is the stronger study. At this point all I can say is I would recommend not hanging everything on Schaalje’s conclusions.

    >>If Dale wants to come here and discuss these probability problems you mention, I’m all ears.

    Me too. Dale has been in poor health for a number of years now and he still manages to put in major hours on several projects he’s got going. I won’t hold my breath, but if he shows up, I’ll pay attention.

    Now I really should be off to bed. All the best!

  9. >>Can you explain how Sidney Rigdon came out as author of the Federalist Papers using Jocker’s method?

    No. Nor would my explanation count for much anyway. I will wait to hear what Craig and the others have to say.

    Why is this not a red flag for you? The Jockers study has been shown to come to false conclusions. I think this questions the validity of his analysis of S/R and the Book of Mormon, because the test not only can be fooled, but was fooled in a demonstration. The false positive issue is a HUGE issue. Shuffling it off for Craig for an explanation is hand-waving a really SERIOUS issue away. I’d love to hear Craig address this. I hope you can acknowledge the seriousness of this issue.

    Furthermore, there is the question about training samples from Jockers. From my post above,

    In order for the NSC method to work, writing samples of test authors are needed. Schaalje et al noted that the Jockers study has sample texts ranging from 114 to 17,979 words, with training texts ranging from 95 to 3752 words. With such a wide disparity of sample texts, the BYU authors indicated that was another problem.

    The measurement of 100 or more word frequencies on texts of less than 100 words is almost sure to produce unreliable measurements (Holmes and Kardos,2003). For the delta procedure, Burrows (2003, p. 21) found that ‘with texts of fewer than two thousand words in length… the test gradually becomes less effective’. Others have worked with texts of 1,000, 5,000, and 10,000 words (Larsen et al.,1980; Hilton, 1990; Holmes, 1992).

    When they’re using a 114 word sample to create a 95 word training set, do you see this as a problem?

    As for the Exodus, I’ll give one more piece of information and let it rest. There is a school of thought that says the Canaanites and Israelites are the same. This school of thought supported by many academics says that the Land of Canaan was not taken over by conquest – rather, the Israelites actually might have been Canaanites who migrated into the highlands and created a new identity for themselves.

    “Joshua really didn’t fight the Battle of Jericho,” said William Dever of the University of Arizona.

    Dever and many others believe Moses and Joshua are fictional, never lived in Egypt, and there was no migration that ever occurred. The Canaanite/Israelites war was a class warfare. Yahweh, the God of Israelites, was a Canaanite deity as well. It was the Israelites that elevated Yahweh above all other Canaanite deities. The Israelites invented Moses, Joshua, Noah etc to give themselves a founding nationality. Basically it was the poor Israelites united against the rich Canaanites. To distinguish themselves, they created myths of coming out of Egypt. There is no evidence to back up one shred of evidence for the Exodus story, or that Moses ever existed, or that there was a migration into either northern or southern Palestine. It also explains why the Israelites had such a hard time letting go of idolatry, since the Canaanite and Israelite religions shared the same source. Canaanites had a pantheon of gods, while Israelites used a Canaanite deity to evolve into a monotheistic religion.

    Food for thought when you have some downtime or get bored from studying S/R… 😉

  10. Roger, I just wanted to add one other thing. I enjoy your passion for this subject. It would be nice to engage you on other topics as well. Certainly you have interests besides Spaulding, right?

  11. MH:

    I’m not in a position to make an informed judgment on the Schaalje/Jockers data at this point. So the details of both studies don’t put up red flags. I would need to study it deeper to determine where or if red flags occur and at the moment I don’t have the time.

    I will say, again, though, that my personal opinion is that you should be careful about hanging everything on Schaalje. If and when it can be demonstrated that his low probabilities are derived as a result of King James English, and when that factor is eliminated the 19th century authors start mingling with BOM chapters on his PCA charts, will you still place so much stock in Schaalje’s study?

    Whether there was a Moses or not, there was certainly a David and Solomon. And accepting on faith that there was a Moses prior to David does not run contrary to other real world evidence. In the absence of data skeptics are free to conclude whatever they want about Moses, but at least they are not free to do that with other historical figures that play key roles in the Bible–the most important of which is Jesus Christ. That is simply not the case with BOM characters.

    Right now I don’t have a lot of other interests, actually, because I don’t have time for other interests. I run a business which demands a lot of time, and I have a family. Studying LDS history, and more specifically BOM history is done in my off hours, which doesn’t leave time for much else except maybe watching the O’Reilly Factor or NCIS every now and then.

    All the best

  12. I should also add that I did not come to the conclusion that S/R best explains the BOM because of Jockers’ study. My conclusion was based on other factors and Jockers’ results merely reinforce that. If Jockers’ results are flawed–and again I don’t think Schaalje has demonstrated that and even Ben Mcguire seemingly agrees when the real author is in the mix–but if they are flawed, then it doesn’t pull the S/R rug out from under me.

  13. It is, unfortunately, destined that scientists usually find evidence that reinforces their previous conclusions.

    http://www.wheatandtares.org/2011/02/10/kuhn-vs-popper-kuhns-challenge-to-popper/

  14. Roger, if the Schaalje study is shown to have legitimate weaknesses, I will definitely look at those. But if I were you, I’d be careful to assume that “Jockers’ results merely reinforce [the Spaulding Theory].” It seems like you should follow you own advice and “be careful about hanging everything on [Jockers].” I know you have other reasons for believing the Spaulding theory. (Of course, I disagree with those reasons too.)

    I’ve exchanged some emails from Dale Broadhurst and Craig Criddle that I would like to add to the conversation. I will break them up into a few comments for readability. I think both of these guys make valid points, and I think their comments really add to the conversation. I’ll start with Dale since he was the first to give me permission to share his email.

    —–

    I read:

    “a controversial study completed by Stanford researchers Mathew Jockers,
    Daniela Witten, and Craig Criddle who concluded that 57% the Book of Mormon
    was authored by Sidney Rigdon and Solomon Spaulding….”

    In a recent exchange with Dr. Jockers, I again see that he is saying that
    the report published by his research team is only a set of RELATIVE
    authorship attributions. That is to say, from among the candidates
    selected for testing, Sidney Rigdon and Solomon Spalding scored
    relatively high percentages in some BoM chapters, via NSC analysis.

    In more recent, updated studies, Jockers added in the Joseph Smith
    word-print, and the adjusted study results are slightly different.

    If Lucy Mack Smith, W.W. Phelps, Brigham Young, etc. had their word-prints
    added to the study, the results would again differ somewhat.

    This is the natural outcome of “closed-set” NSC textual analysis. The
    results will always depend upon the type and amount of data entered.

    The viable argument between Matt’s reporting and Bruce’s more
    recent reporting is NOT the results of “closed set” vs “open set”
    analysis, but rather the viability of selecting certain author-candidates
    for NSC testing in the first place.

    The methodology will only provide 100% certain results if (1) ALL the
    actual authors for a set of texts are included in the word-printing; and
    (2) IF the word-printing itself provides reliable, well differentiated
    analytical tools for textual comparison.

    If these criteria are not met, then the results of NSC testing may provide
    useful leads/hints for further research/investigation, but those results
    will NOT be the sort of absolute proof which would determine a court case
    nor form a definitive new scholarly consensus.

    There is plenty of other ground upon which to argue with the Stanford
    team’s reporting, without accusing them of asserting that Rigdon and
    Spalding undeniably wrote 57% of the Nephite record — that was NOT
    the conclusion of that team’s study report.

    Dale

  15. Dale, if you’re reading this, I would say that is a valid point, and I should have been more careful with how I characterized that particular sentence. There is a bit of nuance that most people don’t usually pay attention to. I would like to note some similar references from the Jockers study itself. Stated as strongly as it is, I hope everyone can see why critics (especially those not educated in statistics) believe the Jockers study “proves” (or in Roger’s case “supports”) 🙂 the Spaulding theory when Jockers et al make statements like this (I bolded a few things here):

    At a macro level, the signals for Rigdon, Isaiah-Malachi, and Spalding dominate the Book of Mormon, with NSC assigning 85% of the text to one of these three candidate authors (delta assigns 93% to these three). Taken together, Rigdon and Spalding account for 57% of the first place assignments and 68% of the second place assignments. Isaiah-Malachi accounts for 28% of the first place assignments and 16% of the second place assignments. Overall only ten chapters lack a signal for Rigdon or Spalding in one of the two most probable positions and four of these ten chapters are chapters known to be derived from the Book of Isaiah, as discussed below. In other words, of the 239 chapters in the Book of Mormon, 229 show either the Rigdon or Spalding signal prominently. Together Rigdon and Spalding receive 64% of the combined first and second place assignments, Isaiah-Malachi receives 21% and other candidates or control authors receive 15% (Fig. 3).

    Jockers et al go on to discuss Isaiah-Malachi at this point.

  16. In a subsequent email, Dale added another piece of advice that I heartily endorse.

    I think that a good case can be made, for the criticism of folks who pick up the Jockers findings, and then trumpet them as proof of Rigdon or Spalding having contributed to the BoM text.

    If the Stanford team’s published report is cited honestly, no such interpretation can be relied upon. It is only when that paper’s data/conclusions are subjected to study, supplement and commentary, that such assertions can honestly be made.

    I can say that my additional study and discoveries make Jockers’ reporting into a compelling argument for 19th century BoM authorship — but I CANNOT say that the Jockers paper, in its own right, should compel readers to accept my conclusions.

    Thus, I suggest that you reserve most of your energy for the purpose of arguing with people who are trying to make Jockers say MORE than he actually reported — and to simply state your reasons for thinking that the original published paper is inadequate for the purpose of firmly establishing Book of Mormon authorship.

    That approach would be perfectly reasonable — and effective.

    Dale

    —-

    Dale, I think that is very well said, and I agree completely. I’ll try to follow this advice.

  17. Ok, I’ve had a few, short emails with Craig Criddle. I asked for his permission to publish the email, and his response was a bit vague, so I will assume that he is leaving it up to my discretion. Here’s exactly what he said when I asked him if I could publish his email.

    —-

    The real problem is that I really, really don’t have time for any substantive on-line engagement – already working practically 24-7.

    It seems a no-win scenario from my point of view. If you publish my comment, and I don’t engage Schaalje and [advocates], it will appear I don’t have a valid response. If you don’t publish it, you’ll be getting lots of uninformed and just plain wrong opinions that declare our study “debunked” .

    I just wanted you to be aware of that weakness.

    —-

    Well, Craig, if you’re reading this, I want to thank you for responding to my email. As I said before, I think you’re leaving this up to my discretion. I am sensitive to your time constraints, and I understand your point of view that this could be a “no-win” situation for you. So, I understand the weakness. I wish you the best in your work. I am trying to give you a fair hearing, and I will do my best not to take your words out of context, or misinterpret them.

    So, here is my question to others (especially Roger): is it “good manners” for me to publish his email (which I will do in the next comment)?

  18. Ok, here is Craig’s email from Feb 9.

    —-

    I am overloaded at work right now, trying to raise money to keep my research program running, and simply cannot invest the time in these on-line discussions, interesting as that might be. The basic argument made against our work — that the true authors could be people we didn’t test– is a straw man. We always acknowledged that a limitation of our work was that we were working with a closed set of authors. And, beyond that, there are serious problems with the method promoted by Schaalje et al. I can’t go into that now, but will likely do so when my work situation becomes more manageable.

    Suffice it to say that I’m confident we got the basic story right. In the time since we published that study, we have found other converging lines of evidence, using different data sets and different methods. The new results support our overall conclusions, and weave together in a consistent narrative.

    Craig

    —–

    Craig followed up on Feb 11,

    —–

    One more thing,

    Schalje relies on a statistical tool to assign posterior probabilities for his Bayesian probabilities, then discounts to zero all of the historical evidence That’s how he comes up with Rigdon as the author of the Federalist paper — by ignoring historical data that implicates Hamilton, and creating a ridiculous closed set of authors that makes no historical sense. His identification of Rigdon simply means that Rigdon’s style is closer to the Federalist papers than that of the other authors in his closed set of authors with no historical connection to the Federalist papers. If posterior probabilities are not adjusted in some way to account for historical data, the results will be flawed, and this is especially the case for a book like the Book of Mormon, a book written in a style that differed from that used by 19th century authors in their known work (i.e., Early Modern English of the King James Bible and Shakespeare).

  19. This second email was very intriguing to me, and I followed up with a question.

    Craig, I appreciate the response. I’m trying to be fair here, and this appears to be a valid criticism, though I’m not sure I understand what you mean “discounts to zero all of the historical evidence”.

    His response was

    What I mean by “discount to zero” is that he places no weight on historical evidence in assigning posterior probabilities.

    I need further clarification. So, at this point, I risk giving an “uneducated” response to Craig’s email, for which I will take responsibility.

    For the sake of the conversation, I want to say that anytime I hear “Bayesian probabilities”, I get a bit nervous. There are 2 kinds of statisticians: Classical, and Bayesian. I’m much more comfortable with classical statistics. Every statistician is required to learn Bayes Theorem, which is a very useful tool for calculating probabilities. Quoting from Wikipedia,

    This theorem is named for Thomas Bayes (pronounced /ˈbeɪz/ or “bays”) and often called Bayes’ law or Bayes’ rule. Bayes’ theorem expresses the conditional probability, or “posterior probability”, of a hypothesis…. Bayes’ theorem is valid in all common interpretations of probability, and it is commonly applied in science and engineering.[2] However, there is disagreement among statisticians regarding its proper implementation.

    While Bayes Theroem is well accepted, Bayes went on to define a whole new field in statistics. I am a classically trained statistician, not a Bayesian statistician. In economics, you have classical economists, and Keynesian economists, and I think it is a similar difference of opinion as to how certain theories of statistics go. Certainly both lines of thought have practical and legitimate uses. But there is going to be a natural amount of disagreement between Bayesian and classical statisticians.

    So, at this point, you can see that this does seem to be a bit of an argument about proper implementation. I would like to look more at this issue of posterior probabilities before I comment further, as I am unclear what Criddle’s argument refers to.

    So let me go back to Craig’s response that Schaalje created “a ridiculous closed set of authors that makes no historical sense” in regards to the Federalist Papers demonstration. At this point, it seems that Schaalje and Jockers are arguing about what constitutes “a ridiculous closed set of authors.” For believers (such as myself) in the Book of Mormon, Jockers also is guilty of “a ridiculous closed set of authors that makes no historical sense” in choosing Spaulding/Rigdon, etc because they weren’t alive when the Book of Mormon was written from 600 BC-400 AD. So in reality, this comes down to a MAJOR disagreement about whether Spaulding or Rigdon are legitimate authors, as well as whether NSC confirms or denies them as major contributors to the BoM. If you believe the BoM is a 19th century creation, then you’re going to support Jockers. If you believe in the divine origin of the BoM, you’re going to support Schaalje.

    So, I can understand why some consider this “a battle of the experts.” But let me state that I still think Schaalje’s study is superior for these reasons: (1) the option of “none of the above”, (2) the “goodness of fit” test, (3) Schaalje’s test was reliability tested against a known author, (4) Jockers used too small of sample texts (114 words) for his training set. Now of course, Jockers can address these improvements in a subsequent paper. As for weaknesses in Schaalje, I do need to study this issue of “posterior probabilities” better to see what the issues are there.

  20. MH:

    >>I am a classically trained statistician, not a Bayesian statistician.

    Well that explains your insight into this. It’s a good thing I did not fall into your trap and instead appealed to the experts! Shame on you, picking on a poor, defenseless, layman! ; )

    >>If you believe the BoM is a 19th century creation, then you’re going to support Jockers. If you believe in the divine origin of the BoM, you’re going to support Schaalje.

    Which is what I have been asserting from the start.

    >>But let me state that I still think Schaalje’s study is superior for these reasons:

    Which is not surprising. As I said, time will tell, but I would not hang my hat exclusively on Schaalje’s peg.

  21. And I would not hanging my hat exclusively on Jocker’s peg (or Broadhurst’s peg) either. 🙂 I will say that my interactions with them have been genuinely respectful and appreciated (as with you.)

  22. Hi MH,
    I have noted Roger’s reliance on the “if the actual author is included in the candidate set, the Jockers method works very well.” However, believing that one of the authors has to be in the candidate set because a person cannot accept the LDS explanation still does not make it a reality.
    On Craig’s complaint that Bruce’s method takes historical data and reduces it to zero is really irrelevant. Craig is perfectly correct that everyone knows that Rigdon did not write any of the Federalist papers. That actually is the point. We do know for a certainty who wrote those papers. That is why they are used as a test bed for people wishing to test new authorship attribution theories.
    Excluding Hamilton from the mix highlighted the problems with a closed set NSC model. As you pointed out, there needs to be a “none of the above” option for the results to have any real meaning, statistically. What the proponents of the original Jockers study have done is impute absolute values from relative results.

    Thanks for allowing my input.
    Glenn

  23. Wow, that took forever for me to read all the comments, and there were only 23. I can go thru 83 comments faster on Wheat & Tares, but I digress.

    I think both studies are flawed and prove nothing. To claim that the Jockers study didn’t have an end goal in mind is disingenuous. Equally, Schaalje had a similar goal…not to refute the BoM, but to refute the Jockers study. It seems both had a conclusion in search of a proof point.

    Sure Schaalje had a few more elements in his study that lends more credibility, but it still proves nothing.

    I don’t know how you can possibly use a word print against a document that wasn’t written in any of the test author’s voice. None of the proposed authors (including Nephi, Alma et al) wrote in KJ English, so how can a word print accurately predict anything?

    I might be naive, but wouldn’t you have to include only texts written in KJ English by the test authors to have acceptable results?

    I say none of this proves anything on either side.

  24. Bishop Rick, how is the Schaalje study flawed? What do you make of the Federalist Papers demonstration? (You’re a bit light on specifics of the flaws.)

  25. I shouldn’t have used the term flawed. I should have simply said that neither study proves anything.
    Don’t you think it impossible to perform a word print study on a text written in KJ’s English when there are no other samples from any of the test authors written in the same fashion?

  26. The Jockers study doesn’t prove who wrote or didn’t write the BoM. All it proved is that IF the author was in the closed group, that it was likely one or both of S/R.

    The Schaalje study doesn’t prove that the results of the Jockers study were wrong. All it proves is that the methodology used in the Jockers study didn’t allow for enough possibilities to be accurate, but it could still be correct. We just don’t know.

    So in my opinion, they were both a waste of time.

  27. Don’t you think it impossible to perform a word print study on a text written in KJ’s English when there are no other samples from any of the test authors written in the same fashion?

    Well, that’s an interesting question. The idea of a wordprint is that each individual has certain uses of words that are unique. So, if a person tries to hide behind KJ English, the wordprint is supposed to still identify them because these word usage patterns are unconscious. So even though Rigdon/Spaulding wrote in KJ English, Jockers thinks these wordprints betray R/S.

    I know that Criddle tried to make some argument above about KJ English, but that seems to undercut his own study. If such a proposition is true, then it calls into question ALL wordprint studies, essentially killing Criddle’s claims along with anybody else who has made a wordprint study. So if Criddle wants to go down that road to question Schaalje, then I think he’s going to have to withdraw his own study as hopelessly flawed. If a wordprint can be faked by using KJ English, then what good are wordprint studies? I can’t see Criddle doing that.

    The Schaalje study doesn’t prove that the results of the Jockers study were wrong.

    You are wrong here, because Jockers results completely contradict Schaalje’s results. let me quote it again.

    Using closed-set NSC, Jockers et al. (2008) attributed 37% of the chapters to Rigdon, 28% to Isaiah/Malachi, 20% to Spalding, 9% to Cowdery, 5% to Pratt, and 1% to Longfellow. In contrast, using open-set NSC, we conclude that 73% of the chapters cannot be reliably attributed to any of the candidate authors.

    This unquestionably disputes Jockers conclusions. I don’t know how you can read that and claim that “Jockers study didn’t allow for enough possibilities to be accurate, but it could still be correct.” Schaalje is saying that Jockers has way too many false positives, and couldn’t possibly be correct in regards to Rigdon or Spalding.

  28. Bishop Rick, let me add one other thing. Both Jockers and Schaalje agree that 28% of chapters were written by Isaiah/Malachi, so their 2 methods agree on that 1 uncontroversial point. But the other 72% Schaalje says can’t be attributed to any author, while Jocker’s claims that 57% of the chapters show a high probability of being written by Rigdon/Spaulding. To use your wording, modified: “The Schaalje study [DOES] prove that the results of the Jockers study were wrong.”

    Jockers is trying to save face and say that in the closed-set, his results are accurate. Ok, yes Jockers is correct. But Schaalje is showing that the closed-set method is highly flawed (via the Federalist Papers demonstration.) He’s come up with an open-set method that is far superior to Jockers closed-set method. I can’t see any useful reason to continue to use Jockers method for any wordprint tests. Schaalje took Jockers method and improved it greatly. Jockers is to be commended for coming up with the method, but Schaalje made MAJOR improvements.

  29. Nice discussion here. I was tangentially involved in the very early stages of Schaalje and Fields’s collaborative effort. My contribution was minimal: I pointed out a reference that addresses the open vs. closed set problem that the authors cited.

    I am very pleased with how it turned out. The Jockers/Criddle study has clearly been surpassed. That isn’t to say that it didn’t make a valuable contribution to the overall discussion.

    I remember research into the KJV emulation question a bit. I recall a study that analyzed whether George Orwell was successful at hiding his authorship of one of the chapters of his book by writing in a different genre or style. Interestingly enough, the conclusion was that he wasn’t.

    Whatever the implications for the Book of Mormon, I expect techniques for proving author x did or didn’t write document y will become more sophisticated in the future. The criminal courts will be interested in how reliable such analysis can become. The related question will be can person z write y and fool the analysis to believing x wrote it by copying his/her style. I think the Schaalje (et al’s) study is a major breakthrough towards answering these kinds of questions, while the Jocker (et al’s) study provides a cautionary tale of getting false positive attributions.

  30. Now it came to pass that in the first year of the reign of the judges over the people of Nephi, from this time forward, king Mosiah having gone the way of all the earth, having warred a good warfare, walking uprightly before God, leaving none to reign in his stead; nevertheless he had established laws, and they were acknowledged by the people; therefore they were obliged to abide by the laws which he had made.

    King Mosiah, a righteous man of God, died without appointing a successor however, the people were left with laws to govern them.

    How can you say that those two statements were written by the same person?
    I realize the second was written by me, but you get my point.

    The BoM was written in such a different style, that I can’t see any wordprint being accurate when it can only compare KJ with 19th Century English. To me this trumps any open vs closed test case.

    72% of inaccurate data still means nothing.

    We agree that the KJ English point does irreparable damage to the Jockers study. Where we disagree is that I don’t think Schaalje proved anything, because its not possible to do so under the circumstances.

    If he were using similar samples in his study and still came up with the numbers he did, I would be onboard, but that did not happen, so I still believe both studies were a waste of time.

    Its obvious what’s going on here. Criddle was trying to prove the BoM was written by S/R by using the method he chose (despite his claims) and Schaalje was trying to do just the opposite.

    I stand by my statement that neither study proved anything. Criddle did not prove the BoM was written by S/R, and Schaalje did not prove it wasn’t.

  31. Bishop Rick, Jockers did mention in a footnote that there are people like Stanley Fish that question wordprint studies in general, so I guess you have some support in that position. I have to say that I somewhat lean in your direction, though I wouldn’t state it nearly as strongly as you have in your previous comment. Here’s some food for thought that may soften your position somewhat.

    Both Jockers and Schaalje correctly identified Isaiah-Malachi. This seems to lend some credibility that wordprints may be able to identify authors in specific situations. Schaalje actually did some reliability testing against a known author in the Federalist Papers, and correctly identified Alexander Hamilton as the author. So there does seem to be some value in Jockers and Schaalje’s studies, and I wouldn’t be so bold to proclaim “both studies were a waste of time” or “neither study proved anything”. I’d trust the Oxford Journal that the studies are more valuable than you are giving them credit.

    As for your demonstration, Schaalje demonstrates that samples need to be >1000 words to be considered legitimate, rather than your 1 sentence example above. Sure, you might fool the computer in a sentence, but the idea in a wordprint is that if you wrote a book, you’re not going to fool the computer (if the method is used properly.)

  32. Here is the paper I remember reading over 2 years ago, but the conclusion is different from what I reported above:

    Multivariate Analysis and the Study of Style Variation
    David Hoover
    Lit Linguist Computing (2003) 18 (4): 341-360. http://llc.oxfordjournals.org/content/18/4/341.abstract

  33. MH,

    Did you really want my example to be >1000 words?
    I was merely making a point.

    Let me be more clear. I think wordprints have merit when, like you say, are performed properly.
    Its obvious that Jockers was not done properly due to being closed.
    And, I don’t think Schaalje had a chance of doing it properly because of the extreme differences in sample styles.

    The fact that both studies identified Isaiah/Malichi tends to back me up.
    There are plenty of like samples from which to identify them from.

  34. Keller,

    I’m not willing to pay to read that article.

  35. Bishop Rick, you seem to have pretty strong opinions without having a background in statistics. I’m just saying I think you might want to soften your stance a bit based on the Federalist Papers. Are you saying Schaalje messed that up?

  36. Actually, you don’t know my background, and I wasn’t referring to the Federalist Papers study at all. I was referring to Schaalje’s open set re-study of the BoM.

    But that said, I do think he messed up the Federalist Papers study for its intended purpose.

    If he really wanted to do a proper comparative study, he would have used a text known to have multiple authors as the BoM is. His study used texts known to come from a single author, so I’m not impressed with the comparison.

    Oh and my opinions on this are probably not as strong as they are coming across to you, but in my opinion, the only thing Schaalje proved was that closed wordprint studies are extremely problematic. He did not prove that NONE of the Jockers study test authors had anything to do with writing the BoM.

  37. You’re right, I don’t know your background. But I haven’t heard you attack any of the statistical methods directly. Sorry if I was mistaken, but your comments so far don’t betray you as a statistician. Are you a statistican?

  38. No, but I play one on the internet.

    I have had stats classes that I did well in and was a calculus tutor in college. Have an advanced degree that included extensive macro and micro economics. Math is not above me.

  39. A few random thoughts:

    1. Criddle commented that “Schalje relies on a statistical tool to assign posterior probabilities for his Bayesian probabilities, then discounts to zero all of the historical evidence.” This is an odd statement, and it’s also completely backwards. A turned-around version of this statement is our main problem with the Criddle study, and is the very reason we developed the open-set extension.

    By using closed set methods with only the S/R suspects as candidate authors, Criddle discounted to zero all authorship possibilities other than his S/R candidates. The open-set method — with nonzero prior probabilities for all candidate authors — by contrast, does not discount anyone or anything (including Criddle’s controversial historical evidence). In the paper, we formally assigned equal prior authorship probabilities to all of the candidate authors, including Spalding and Rigdon. For fun, I just reran the open-set NSC attributions with prior probabilities set as follows: early Rigdon .4, late Rigdon .1, Spalding .2, Cowdery .14, Smith .13, Pratt .01, Isaiah .01, someone else .01. The final attributions were almost identical to the equal prior situation. This is because the data so strongly contradicts the S/R prior.

    Criddle’s closed set method is equivalent to open-set NSC with equal nonzero priors for his S/R candidates, but priors forced to zero for everyone else. Criddle and Jockers are right about their own study: if you force the S/R theory to be true (by using closed set NSC model), NSC classification will generate results consistent with the S/R theory. But the crippling circularity of this reasoning renders their results inconsequential (alliteration inspired by Neal A. Maxwell).

    2. Criddle said: “there are serious problems with the method promoted by Schaalje et al. I can’t go into that now, but will likely do so when my work situation becomes more manageable.” This smacks of false bravado. I will wait with interest for his detailed comments on this topic, especially since the open-set method is simply a generalization of the closed-set NSC method used by Criddle himself. But seriously, if there is something wrong with our generalization that is obvious to Criddle, I would like to know. (It’s certainly possible that we have missed something. Academic discussions about these kinds of things are what the professional literature is all about.)

    3. Criddle and Jockers keep saying that our argument was a straw man argument. I just don’t see it. We provided literature citations for the importance of allowing the candidate set to be open, we extended the Criddle method to allow an open set, we pointed out empirical problems with ignoring text size (a point that Criddle and Jockers admitted being worried about), and we constructed an artificial attribution example to demonstrate that nothing is lost but much is gained from properly allowing the candidate set to be open. The point of the artificial Hamilton-Rigdon example was only that absurd results could be avoided using the open-set technology. We included all of the Criddle candidate authors in our study with nonzero prior probabilities, and we used exactly the same marker words. I think that the cry of ‘straw man’ is too often used in place of well-considered counter-arguments.

    4. Bishop Rick (by the way, our first counselor in a bishopric a few years ago was named Rick, and because the Bishop was gone so often, was referred to by most of the ward members as Bishop Rick): It seems to me as well that “Criddle was trying to prove the BoM was written by S/R . . . despite his claims.” We were trying to bring some sanity to the situation. We didn’t prove that the S/R theory is false – we simply argued that Criddle’s methods were flawed, so his results are meaningless. In doing so, we found that the NSC method could be made more useful and powerful in authorship attribution problems. We admitted in the Conclusion Section that more work is needed; this is not the final stylometric word in Book of Mormon research.

    I found the entire project a fascinating application of statistical methods, independent of the Book of Mormon connection. So for me, the project was anything but a waste of time. Jockers has moved on to comparisons of other machine-learning tools for authorship attribution, so I don’t think he sees this project as a waste of time either.

    5. We made a small effort at dealing with the archaic language issue. As mentioned in the paper, in one of our analyses we dropped “and it came to pass that” from the Book of Mormon chapters (we actually pretended that it was one word, effectively dropping it from the texts). The classification results were almost identical. But clearly, more needs to be done.

    6. Keller: Thanks for your initial help on the project. As I remember, you pointed out the Koppel paper, a great and timely review of the authorship attribution literature.

  40. I formally retract my “…waste of time…” comment. MH knows that I often make statements sound stronger than intended and he gives me opportunities to soften my stance…sometimes I take him up on those opportunities.

  41. Bruce, I am honored that you joined the discussion. THANK YOU!!! I joined the Mormon Discussions topic, but when Roger and Dale turned it into another rehash of the Spaulding Theory, I lost interest. I never understood Criddle’s statement about a “statistical tool to assign posterior probabilities for his Bayesian probabilities”. Thanks for commenting. I will probably post a revised version of this post over at Wheat and Tares on Monday. Perhaps you could monitor that if you’re interested. I think this is a very important topic.

    Keller, thanks for the link to the abstract. I’d like to read it too, but I already buy too many books, so I don’t want to pay to read it either. It sounds like an interesting topic, that would really add to the discussion. I was pleased to download Schaalje’s manuscript for free–I wish that other document could be downloaded for free as well.

    Bishop Rick, I’m glad to know your background a little bit. Robert Riech has a book about the economy and references LDS banker Spencer Eccles as part of the New Deal that I found interesting. (I’ll probably post on it in a month or 2.) Do you lean towards Keynesian or classical economics?

  42. That’s an interesting question. I don’t think economics is that black and white. An example is how Keynesian economics worked to pull us out of the great depression, but I don’t think have done anything for our recent economic situation. During the depression the unemployed labor force was blue collar. Investing in the infrastructure made sense. Today the economy is driven by white collar workers. Investing in the infrastructure did nothing for the white collar worker. The trickle down effect stayed in the blue collar sector. Cash for clunkers just created a temporary spike in the auto industry while removing good mid-level used autos from the market, making it difficult for fiscally conscious buyers to stay out of debt. Also, I don’t believe in too big to fail. On the flip side, I don’t believe that supply creates its own demand. Just as in politics, I am a moderate. I think there is good and bad in both schools of thought.

  43. I have a question about this whole Spaulding manuscript issue that I can’t seem to find satisfactory answer to and wanted to post it here. Please excuse my post if completely inappropriate!

    From my search or study for information, of the two manuscripts attributed to Spaulding, one was had in Joseph Smith’s time (found to have no resemblemce) and the other manuscript, Manuscript Found (or Manuscript Story), which has some similarities, wasn’t published until much later, 1886 if my information is accurate.

    My question is, how does someone claiming the Book of Mormon was based on this other manuscript not see that this second manuscript could simply have been written later, mixing then-known information from the Book of Mormon into some other story, thus producing a similar work? Where is evidence that this 1886 publication actually came before the Book of Mormon was published?

    Thanks for any information you provide!

  44. David, I’ve written several posts on the Spaulding Theory: (1) Debunking it, (2) Manuscript Found Part 1, and (3) Part 2, so you can learn a lot about it if you’re interested.

    To briefly answer your question, only 1 manuscript has ever been attributed to Spaulding. Spaulding died in 1816. A man by the name of Doctor Philastus Hurlburt (Doctor was his name, he was not a physician) talked with Spaulding’s widow in the late 1830’s and found an unnamed manuscript about a group of Romans shipwrecked in America at the time of Constantine. Hurlbut knew it was a bust, but reported other findings, and gave the manuscript to E.D. Howe. Nobody knew where it was after that. In 1885, it was discovered, and given to the RLDS church and published under the name “Manuscript Found.” In Part 1 and 2 of the posts I listed above, I give an idea of how terrible Spaulding’s grammar is and unintentionally funny the novel is.

    Spaulding proponents have suggested that Spaulding must have written another manuscript, because there is obviously no plagiarism in “Manuscript Found.” Dale Broadhurst (THE current Spaulding expert), believes in a 2nd unknown manuscript, but believes “plagiarism” is too strong of a word. He thinks Spaulding’s novel may have given Joseph ideas for the Book of Mormon, but Joseph didn’t actually copy “Manuscript Found.”

    “Manuscript Found” was definitely written by Spaulding, who died in 1816, so there is nobody that doubts the authenticity of the document published by the RLDS church 70 years after his death.

  45. I have gone through the information and have another question. Is the “Manuscript Found” published 70 years later the same account Hurlbut received from Spaulding’s widow? It seems to me that the later publication was added upon to more closely match the Book of Mormon account, based on your posts.

    How do we know it wasn’t changed to more closely match the Book of Mormon and then published, in support of the theory? It would seem Hurlbut would have found the similarities if it was one and the same. Are we sure it’s the same work?

  46. The reason I ask, is because there are so many similarities in the 1886 version that it seems odd that nobody would have found them in the earlier, original version. Here is a site that details this:
    http://solomonspalding.com/docs2/vernP1.htm#pg10

    It kind of seems like the obvious question that everyone else is assuming, that this later version is the same as the earlier one, but certainly there would have been opportunity to make this later version similar and then have it come forward.

  47. David, I see you are acquainted with Dale Broadhurst’s website. Have you actually read the Manuscript Found, or the links I posted? Your comment #46 leads me to believe you haven’t. First of all, Manuscript Found was discovered in Hawaii, far from Solomon’s home of Pittsburgh, PA.

    Let me quote from Part 1:

    LL Rice purchased the assets of the Painesville Telegraph in 1839-40. In 1885 or so, he looked through the assets and discovered Spaulding’s Manuscript. The manuscript was donated to Oberlin College after being discovered in Hawaii.

    Pages 3-11 tell how the document came into the hands of Oberlin College, and has letters to Joseph Smith III (Joseph’s son), who was ordained prophet of the RLDS church on April 6, 1860. Apparently the RLDS church published the manuscript sometime around 1885. Some interesting quotes from these pages start on page 5-6. The document was discovered in Hawaii by Rice who was a friend Fairchild, president of Oberlin College in Ohio. Many people wanted to claim the manuscript, but they felt it best to offer it to Joseph III, since he is the son of Joseph Smith. I have [bolded] some points I find interesting. Let me quote from pages 5-6,

    “There seems to be no reason to doubt that this is the long-lost story. Mr. Rice, myself, and others, compared it with the Book of Mormon, and could detect no resemblence between the two, in general or in detail. There seems to be no name or incident common to the two. The solemn style of the Book of Mormon, in imitation of the English Scriptures, does not appear in the manuscript. The only resemblance is in the fact that both profess to set forth the history of the lost tribes. Some other explanation of the origin of the Book of Mormon must be found, if any explanation is required.”

    Signed, James H. Fairchild.

    My advice is to read the actual manuscript yourself (here is the link to a digital copy from Oberlin College) before you fall for Dale’s “coincidences.” I think you’ll readily see that Dale is making more out of these coincidences than is really there.

    Here is a funny quote from the book (Part 2). Please tell me if this sounds like the Book of Mormon (note spelling is found exactly as Spaulding wrote it):

    Seven young women we had on board as passenjers to viset certain friends in Brittian. Three of them were ladies of rank & the rest were healthy bucksom lassies. Whilst deliberating on this subject a mariner arose whom we called Droll Tom Hark ye, shipmates says he. Whilst tossed on the foaming billows what brave son of Neptune had any more regard for a woman than a Sturgeon, but now we are all safely anchored on Terra firma, our sails furled & ship keeled up, I have a huge longing for some of those rosy dames. But willing to take my chance with my shipmates, I propose that they should make their choice of husbands. The plan was instantly adopted. As the choice fell on the young women they had a consultation on the subject, & in a short time made known the result. Droll Tom was rewarded for his benevolent proposal with one of the most sprightly, rosy dames in the company. Three other of the most cheerful, resolute mariners were chosen by the other three bucksom Lassies. The three young Ladies of rank fixed their choice on the Captain the Mate & myself. Happy indeed in my partner, I had formed an high esteem of the excellent qualities of her mind The young Lady who chose me for a partner was possessed of every attractive charm both of body & mind. We united heart & hand with the fairest prospect of enjoying every delight & gratification which are attendant on the connubial state. Thus ended the affair. You may well conceive our singular situation. The six poor fellows who were doomed to live in a state of celibacy or accept savage dames, discovered a little chagrin & anxiety. However they consoled themselves with the idea of living in families, where they would enjoy the company of the fair sex, & be relieved from the work which belongs to the department of women.

  48. Ha ha, good stuff. Don’t get me wrong, I don’t believe in the Spaulding manuscript. But there are certainly quite a bit more similarities in “Manuscript Found” than appear on the surface. My personal theory is that what Hurlbut reviewed wasn’t this “Manuscript Found” but a simpler, similarity-free account (which explains why in his complete obsession to find something to hold onto, he couldn’t). I think someone tried to make additional similarities (changed the names, same ideas) and then this newer version was published. There certainly was motive AND opportunity.

    It’s just funny how a document published in 1886 is even a consideration. Can you imagine trying to use something like this against opponents of the Church? “Yeah, so, we know it came A LONG TIME after but it’s good, after all. And what’s in it proves that . . . . etc., etc..” Wouldn’t work in a million years.

    @MH

  49. I found a way were i can check out the viabilty of Jockers study for my self. there is the program that is freely available called signature. You just load the texts that you want to compare into the program. then select them and the program compares them. the wensitr were the program can be downloaded has a powerpoint presation explaining how to use the program.

  50. David, I’m not sure why you’re making an issue about this being published in 1886. I know you’re making he case that this might not be Hurlbut’s document, but it seems to match the description quite well, IMO. The grammar and spelling are horrendous, and the story is just plain funny. Seriously, you should read it! (It’s not very long.) I don’t see how anyone can read this and think the it is remotely similar to the Book of Mormon.

    Perhaps there is another manuscript–until one surfaces, we will never know. But my belief is that this novel was soooo bad, that it wasn’t worth publishing. I don’t buy Dale’s arguments that this is similar to the Book of Mormon. For one thing, Manuscript Found is approximately 1/3 as long as the Book of Mormon, so Joseph had to come up with a lot of new material. Secondly, the Book of Mormon discusses advanced civilizations with chariots, palaces of gold, etc, while Manuscript Found talks of teepees, wigwams, and really uncivilized Indians who fight over feathers and headresses. The Ohons (as they are called) are primitive, not advanced like Nephites. And I don’t recall “Droll Tom”, or “bucksom lassies” ever mentioned in the Book of Mormon. Read the book yourself–don’t rely on Dale’s word. I thought the book was rather funny.

    I’ve been intending to write a synopsis of each chapter, and discuss the characters, such as Fabian to show how different the names are. Perhaps I will come up with something soon, but I have sooo many ideas right now on other topics.

Comment navigation

← Older Comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: