Category Archives: junk science

When Benford’s Law Breaks Down

Can you use Benford's Law to reliably detect election fraud?

Benford's Law Leading Digit Distribution

This question has been raised often since the U.S. elections on 3 November 2020. The earliest mention of it we can find where it was used to question results in this election dates to 6 November 2020, at this Twitter thread, which featured this infographic looking at the city of Milwaukee's election results in the U.S. presidential race. That was followed by tweets looking at vote tallies in Michigan, Chicago, Allegheny County (Pittsburgh), and other cities and states with long reputations for election irregularities.

Much of the analysis behind these posts appears to have originated with Rajat Gupta's DIY Election Fraud Analysis Using Benford's Law tutorial from September 2020, which provides a guide for how to apply Benford's Law to evaluate election results using spreadsheet applications like Microsoft Excel.

As such tutorials go, it is relatively well done, providing step by step instructions that anyone with basic spreadsheet programming experience can follow to conduct their own analysis to see if their dataset of interest follows Benford's Law and generate a chart showing the outcome. But the tutorial has a huge problem, in that it skips over a huge factor that determines whether Benford's Law can even be successfully applied to evaluate whether the data represents the results of a natural outcome or has potentially been artificially manipulated.

To be successfully applied to detect potential fraud, the dataset must contain values that span several orders of magnitude. In the case of election results from voting precincts, the number of voters in individual precincts would have to number from the 10s, to the 100s, to the 1,000s, to the 10,000s, and so on.

But when you look at the precinct voting data covering the geographies in question, it most often covers data that predominantly falls within a single order of magnitude. This limitation makes Benford's Law an unreliable indicator for detecting potential fraud in this application.

Matt Parker, avid student of math gone wrong and currently the #1 best selling author in Amazon's Mathematics History subcategory, has put together the following video to explain why Benford's Law doesn't work well under these limiting circumstances. It is well worth the 18 minute investment of time to understand where Benford's Law can be successfully applied to detect potential fraud and where it cannot.

Benford's Law has recently been more successfully applied to challenge the validity of the number of coronavirus infections being reported by several nations. The difference in determining its success comes down to the virus' exponential rate of spread, which quickly generates an escalating number of cases that satisfies the requirement that the range of values subjected to analysis using Benford's Law span several orders of magnitude.

With that condition unmet for the raised election examples, we find that using Benford's Law to detect potential voter fraud is unreliable. If the authors of the analyses identifying voter fraud were aware of the deficiency of data used to support their findings, we would categorize these results as the product of junk science. We don't however think that harsh assessment applies for these cases, as most seem to be following an otherwise useful guide that omits presenting the conditions that must be satisfied to properly apply Benford's Law. That makes this situation different from other examples of junk science where the offenders clearly knew their data's deficiencies and failed to disclose them, sometimes with catastrophic effect.

We'll close by providing a short guide to our work covering related aspects of the topics we've discussed and pointing to academic work that provides more background.

Previously on Political Calculations

References

Deckert, Joseph; Myagkov, Mikhail; and Ordeshook, Peter C. Benford's Law and the Detection of Election Fraud. Political Analysis, Volume 19, Issue 3, Summer 2011, pp. 245-268. DOI: 10.1093/pan/mpr014.

Mebane, Walter R. New Research on Election Fraud and Benford's Law. Election Updates. [Online Article]. 23 August 2011.

Mebane, Walter R. Inappropriate Applications of Benford's Law Regularities to Some Data from the Presidential Election in the United States. [Online Article (PDF Document)]. 10 November 2020.

Brown, Michelle. Does the Application of Benford's Law Reliably Identify Fraud on Election Day? Georgetown University Graduate Theses and Dissertations - Public Policy. [PDF Document]. 2012.

The Scariest Prime Number

Der Hl. Augustinus und der Teufel - Source: http://sprott.physics.wisc.edu/pickover/pc/1000000000000066600000000000001.html

Prime numbers can be very strange. One is believed by some to be an embodiment of evil.

That number is known as Belphegor's prime. Named after the demon Belphegor, one of the seven princes of Hell, this prime number is a numeric palindrome, which means it will read the same if you write its 31 digits either forwards or backwards.

1000000000000066600000000000001

If you don't look at it too closely, you might dismiss it as a harmless binary number, if not for the Number of the Beast, 666, which itself is the sum of the squares of the first seven prime numbers, lurking within the middle of it.

As it is, you would write Belphegor's prime by starting with the number 1, followed by 13 zeroes, then the aforementioned Number of the Beast, then another 13 zeroes before ending again with the number 1. If you wanted to convert it into binary format, it is a much less interesting 100 digits worth of 1s and 0s. The true terror of it must be experienced in decimal form.

And yes, it is indeed a prime number that may not be divided evenly by any other whole number other than itself or 1. Check it out for yourself, if you dare!

But if not, at least you now know how to terrify a mathematician on the next Halloween by unleashing the ugly specter of numerology.

Cut from the Same Psychological Cloth

"Scientific misconduct and sexual assault have more in common than you might think."

That's UCLA's Michael Chwe's provocative introduction to an April 2016 article at Retraction Watch, in which he explored a number of predatory behaviors that appear to be disturbingly common among both alleged junk scientists and individuals engaged in sexual harrassment.

For starters, in both scientific misconduct cases and sexual assault or sexual harassment cases, typically a weak person accuses a powerful person. The accused is usually much closer to administrators and the investigative system: the first administrator to confront Diederik Stapel with charges that he had faked his data played tennis with Stapel every Wednesday, and told him that he would “like nothing better” than to believe in Stapel’s innocence. After an investigation found that Sujit Choudhry, dean of the UC Berkeley Law School, sexually harassed executive assistant Tyann Sorrell, UC Berkeley Provost Claude Steele told Sorrell that he would not fire Choudhry because it might “destroy his future chances for higher appointment.”
In contrast, people who report scientific fraud and sexual assault and harassment typically do not have powerful friends, and decide to report at great personal cost. The students of Elizabeth Goodwin abandoned many years of graduate school study in order to report that Goodwin had faked data. As Sarah LaMartina told Science in 2006:
We kept thinking, ‘Are we just stupid [to turn Goodwin in]?’ . . . Sure, it’s the right thing to do, but right for who? . . . Who is going to benefit from this? Nobody.
LaMartina lost her appetite and fifteen pounds, while faculty members in the department all supported Goodwin. Similarly, sexual assault is one of the least reported crimes: In a 2015 survey of 23,000 college students, only 12.5 percent of rape incidents and 4.3 percent of sexual battery incidents experienced by women were reported to authorities. People reporting sexual assault face the possibility of violent reprisals, not being believed by authorities, and even being blamed for the assault.
A particularly insidious dynamic in both scientific fraud and sexual assault and harassment is that people who report it are made to feel that they themselves are implicated in the offense. In a 2006 survey published in the Journal of Interpersonal Violence, the vast majority of women who were raped did not report to authorities; the most common reason they provided (63 percent) was that they themselves would be blamed for the assault. Similarly, as pointed out in a 2010 editorial by the journal Nature:
A young scientist’s reputation is tethered to the successes and failures of his or her adviser, and when that adviser is accused of misconduct, trainees can also be viewed with suspicion.
Even for an established scientist, accusing another of fraud can be very costly. In her book about Jan Hendrik Schön, Eugenie Samuel Reich writes:
A whistleblower of scientific fraud once told me that he felt he needed the right to remain anonymous for the rest of his life. “Like a rape victim,” he said. . . . Thinking of a fraud allegation as if it were an allegation of sexual abuse, I could start to understand on an instinctive level why scientists might feel strongly and yet be very fearful about coming forward.

One of the things that stands out immediately is the extent to which the individuals who engaged in scientific misconduct exploited their personal connections and positions of authority to relentlessly attempt to denigrate the reputations and well-being of the whistleblowers who came forward to challenge their reigns of personal abuse.

Beyond that, it is very interesting to consider the behavioral similarities and parallels between those called out for having engaged in junk science prior to those being called out in today's scandal-ridden headlines involving the power-elite of both Hollywood and Washington D.C. Especially when those stories involve the unethical and systematic actions taken by the abusive personalities against the people who were willing to blow the whistle against their misconduct.

Consider the multiple stories that have come to light involving the recently convicted USA Gymnastics doctor Larry Nassar or of Hollywood's Harvey Weinstein, the latter of whom reports indicate had his own personal "enemies list" of people he targeted for special and ongoing abuse.

The Observer has gained access to a secret hitlist of almost 100 prominent individuals targeted by Harvey Weinstein in an extraordinary attempt to discover what they knew about sexual misconduct claims against him and whether they were intending to go public.
The previously undisclosed list contains a total of 91 actors, publicists, producers, financiers and others working in the film industry, all of whom Weinstein allegedly identified as part of a strategy to prevent accusers from going public with sexual misconduct claims against him.
The names, apparently drawn up by Weinstein himself, were distributed to a team hired by the film producer to suppress claims that he had sexually harassed or assaulted numerous women.
The document was compiled in early 2017, around nine months before the storm that blew up on 5 October when the New York Times published a series of sexual harassment allegations against Weinstein.

Weinstein's enemies list was used to support the gathering of damaging information on his alleged victims and potential whistleblowers, which in turn, was planned to be used to both distract attention away from and to discredit any statements they might make about his personal conduct, to wreck their professional and personal reputations for the purpose of damaging their career prospects as a means to punish them for standing up to him, and to send a message to others about what might happen to them if they ever went public with allegations of misconduct on his part.

What kind of person is like that?

What kind of person engages in serial episodes of misconduct, whether it be cooking data to obtain predetermined results or engaging in acts of sexual harrassment or assault, then puts together an enemies list to facilitate their ability to further demean and diminish or to stalk and terrorize the targets of their personal abuse?

In our Examples of Junk Science series, we paid special attention to the kind of personality traits that characterized the unethical and ultimately antisocial and toxic behaviors that were common among confirmed pseudoscientists. As a hypothesis, we think that many of these are the same traits and characteristics associated with having a narcissistic personality disorder, which is indicated by their having five or more of the following symptoms of the spectrum condition:

Exaggerates own importance
Is preoccupied with fantasies of success, power, beauty, intelligence or ideal romance
Believes he or she is special and can only be understood by other special people or institutions
Requires constant attention and admiration from others
Has unreasonable expectations of favorable treatment
Takes advantage of others to reach his or her own goals
Disregards the feelings of others, lacks empathy
Is often envious of others or believes other people are envious of him or her
Shows arrogant behaviors and attitudes

Psychology Today drills down deeper into each of these symptoms to provide more insight into how each might manifest itself (highly recommended reading, particularly if you ever have to deal with such a person and their abnormal behavior with any regularity).

Dealing with such a personality however can be an especially difficult challenge, because often, their go-to strategy to get away with their misconduct involves launching an unending smear campaign or other forms of social menacing against those who either have or who might expose their misconduct.

Most smear campaigners are highly narcissistic, and narcissists cannot ever be expected to apologize, come clean or admit any wrongdoing, even if caught red-handed in their lies. They truly believe, in their own way, that a smear campaign is the right thing to do to you, because you have opposed them, and you should have known better than to do such an unthinkable thing, so it’s simply all your fault they’re smearing you anyhow. They’re teaching you a lesson — agree with whatever they want, or else. You “asked for it”, and they’re teaching you better.
Smear campaigners are like spoiled playground bullies who kick another child when the teacher’s back is turned, just because the child doesn’t give them whatever they want. They cannot be made to empathize, and they are well-practiced in their abusive games, because they have been playing them all their lives.

In the quoted passage above, we've emphasized the first sentence with red boldface font because it largely agrees with the kind of behavior that we've observed on the part of several of the pseudo-scientists whose egregious academic misdeeds were referenced during our Examples of Junk Science series back in 2016, where they have not acknowledged the serious deficiencies that led to their work being dismissed as little more than junk science in the first place.

In place of that, we've often seen really unprofessional behaviors involving anything from simple namecalling on through to prolonged smear campaigns directed at their perceived enemies with the apparent intent of damaging their credibility, where new frenzies of smears often seem prompted by little more than the most minor of offenses that only they perceive.

How do you deal with that if you're on the receiving end of that kind of personal abuse?

Throughout this article, we've used the term "enemies list" rather than "hit list" because it harkens back to the days of Richard Nixon, whose paranoia drove the biggest political corruption scandal in the U.S. during the twentieth century, or more importantly, of Joseph McCarthy, whose enemies list had greater impact and whose name has become synonymous with unethically-inspired smear campaigns, thanks to what has been described as his "driven, sanctimonious, dogmatic behaviors that are hallmarks of narcissism with an authority attachment". Sadly, in our experience, the twenty-first century's pseudoscientists and sexual harassers would very much appear to be cut from the same psychological cloth, so if you ever come across someone who has a similar enemies list that they're so worked up about that they're acting out on it by engaging in a smear campaign, you'll have a pretty good idea of what to expect from that kind of personality.

Amy Tuteur, a critic and target of the pseudoscience anti-vax movement, has a unique perspective of what it means to make the enemies list of such people, which we've excerpted the following two points below (she makes a third point that's more specific to what she's encountered with the anti-vax "movement", but the following two points are applicable much more generally):

An enemies list is an implicit acknowledgement that the facts are not on their side.
The purpose of the list is to preemptively exclude list members from the echo chambers that are so vital to the propagation of pseudoscience.

We would suggest following a strategy that we hope would be an improvement on how Dwight Eisenhower dealt with Joe McCarthy's similarly unethical and unprofessional behavior, where ultimately, the narcissistic and paranoid McCarthy destroyed his own credibility with his obsessive conduct, which he either could not or would not constrain. Given sufficient time, the narcissistic pseudo-scientist or sexual harasser will publicly hang themselves with the rope from what is most often their own one-way campaign of abuse, where all you need to do as a target is to hang tough and act behind the scenes to limit the damage they might do as much as possible until that happens (Eisenhower's approach to the politically-powerful McCarthy), while making their ongoing unethical conduct as visible as possible to as many people as possible (the improvement), which makes it happen faster.

Whether they be pseudo-scientists or sexual harrassers, when the narcissistic misconduct and targeted abuse in which they've engaged becomes widely known - when everybody knows - is when the end is near for them. When all can see what they're doing and what they've done, and when no one sees them or their conduct as anything other than contemptible, is when they lose whatever power they had to intimidate others to get away with their misconduct without consequences.

Just follow the stories of the falls of Harvey Weinstein or of Joseph McCarthy to see how that works.

Image credits: New York City Media and Library of Congress.

Postscript

Today (9 February 2018) is the 68th anniversary of the date that Senator Joseph McCarthy launched his "Red Scare" campaign. Coincidentally, 9 February 2018 marks the date that the Los Angeles Police Department referred three cases of sexual abuse allegedly involving Harvey Weinstein to the Los Angeles District Attorney. We had originally planned to present this post as our only Friday feature, but bumped it since we've been focusing on covering the turmoil in the U.S. stock market this past week.

Honesty as a Path for Avoiding Failure

When a scientist realizes that they've made a fundamental error in their research that has the potential to invalidate their findings, they are often confronted with an ethical dilemma in determining what course of action that they might take to address the situation. Richard Mann, a young researcher from Uppsala University, lived through that scenario back in 2012, when a colleague contacted him right before he presented a lecture based on the results of his research that he had a problem that called his results into question.

When he gave his seminar, Mann marked the slides displaying his questionable results with the words "caution, possibly invalid". But he was still not convinced that a full retraction of his paper, published in Plos Computational Biology, was necessary, and he spent the next few weeks debating whether he could simply correct his mistake with a new analysis rather than retract the paper.
But after about a month, he came to see that a full retraction was the better option as it was going to take him at least six months to wade through the mess that the faulty analysis had created. However, it had occurred to him that there was a third option: to keep quiet about his mistake and hope that no one noticed it.
After numerous sleepless nights grappling with the ethics of such silence, he eventually plumped for retraction. And looking back, it is easy to say that he made the right choice, he remarks. "But I would be amazed if people in that situation genuinely do not have thoughts about [keeping quiet]. I had first, second and third thoughts." It was his longing to be able to sleep properly again that convinced him to stay on the ethical path, he adds.

Mann's case represents a success story for ethics in science, where his choices to demonstrate personal integrity and to provide transparency regarding the errors he had made through the retraction of his work proved to have no impact on his professional career, though he may have feared it. Such are the rewards of integrity and transparency in science, where the honest pursuit of truth outweighs both personal reputation and professional standing.

Still, an 2017 anonymous straw poll of 220 scientists indicated that 5% would choose to do nothing if they detected an error in their own work after it had been published in a high-impact journal, where they would hope that none of their peers would ever notice, while another 9% would only retract a paper if another researcher had specifically identified their error.

According to Nature, only a tiny fraction of published papers are ever retracted, even though a considerably higher percentage of scientists have admitted to knowing of issues that would potentially invalidate their published results in confidential surveys.

The reasons behind the rise in retractions are still unclear. "I don't think that there is suddenly a boom in the production of fraudulent or erroneous work," says John Ioannidis, a professor of health policy at Stanford University School of Medicine in California, who has spent much of his career tracking how medical science produces flawed results.
In surveys, around 1–2% of scientists admit to having fabricated, falsified or modified data or results at least once (D. Fanelli PLoS ONE 4, e5738; 2009). But over the past decade, retraction notices for published papers have increased from 0.001% of the total to only about 0.02%. And, Ioannidis says, that subset of papers is "the tip of the iceberg" — too small and fragmentary for any useful conclusions to be drawn about the overall rates of sloppiness or misconduct.

There is, of course, a difference between errors resulting from what Ioannidis calles "sloppiness", which can run the gamut from data measurement errors to the use of less-than-optimal analytical methods, which can all happen to honest researchers, and those that get baked into research findings through knowing misconduct.

The good news is that for honest scientists who act to disclose errors in their work, there is no career penalty. And why should there be? They are making science work the way that it should, where they are contributing to the advancement of their field where the communication of what works and what doesn't work has value. As serial entrepreneur James Altucher has said, "honesty is the fastest way to prevent a mistake from turning into a failure."

The bigger problem is posed by those individuals who put other goals ahead of honesty. The ones who choose to remain silent when they know their findings will fail to stand up to serious scrutiny. Or worse, the ones who choose to engage in irrational, hateful attacks against the individuals who detect and report their scientific misconduct as a means to distract attention away from it, which is another form of refusing to acknowledge the errors in their work.

The latter population are known as pseudoscientists. Fortunately, they're a very small minority, but unfortunately, they create outsized problems within their fields of study, where they can continue to do damage until they're exposed and isolated.

Previously on Political Calculations

The Rewards of Integrity and Transparency

One week ago, the Wall Street Journal broke the news that the World Bank had a serious problem with one of its most popular and useful products, its annual Doing Business index.

The World Bank repeatedly changed the methodology of one of its flagship economic reports over several years in ways it now says were unfair and misleading.
The World Bank’s chief economist, Paul Romer, told The Wall Street Journal on Friday he would correct and recalculate national rankings of business competitiveness in the report called “Doing Business” going back at least four years.
The revisions could be particularly relevant to Chile, whose standings have been volatile in recent years—and potentially tainted by political motivations of World Bank staff, Mr. Romer said.
The report is one of the most visible World Bank initiatives, ranking countries around the world by the competitiveness of their business environment. Countries compete against each other to improve their standings, and the report draws extensive international media coverage.

In the days since, Romer has clarified that he doesn't believe that the World Bank staff engaged in a politically-motivated strategy aimed at disadvantaging Chile's position within its annual rankings, but instead failed to adequately explain how changes that the World Bank's staff made in updating their methodology of its Doing Business index affected Chile's position within the rankings from year to year.

Looking at the controversy from the outside, we can see why a political bias on the part of the World Bank's staff might be suspected, where positive and negative changes in Chile's position within the annual rankings coincided with changes in the occupancy of Chile's presidential palace.

That's why we appreciate Romer's willingness to openly discuss the methods and issues with communication, including his own, that have contributed to the situation. In time, thanks to the demonstrations of integrity and transparency that Romer is providing today as the staff of the World Bank works to resolve the issues that have been raised, the people who look to the Doing Business product will be able to have confidence in its quality. That will be the reward of demonstrating integrity and providing full transparency during a pretty mild version of an international public relations crisis.

Update 27 January 2018: Perhaps not as mild an international public relations crisis as we described. Paul Romer has stepped down as Chief Economist at the World Bank. This action is likely a consequence of the damage to the World Bank's reputation that arose from his original comments to the Wall Street Journal suggesting that political bias may have intruded into the Doing Business rankings to Chile's detriment.

Analysis: The problem for the World Bank now is that the issues that Romer identified with its methodology for producing the Doing Business index predate his tenure at the institution. How the World Bank addresses those issues will be subject to considerable scrutiny, where the institution will still need to provide full transparency into its methods for producing the index in order to restore confidence in its analytical practices. Without that kind of transparency, the issues that Romer raised, including the potential for political bias that Romer indicates he incorrectly stated, will not go away.

Not every matter involving correcting the record has the worldwide visibility of what is happening at the World Bank. We can find similar demonstrations of the benefits of integrity and transparency at a much smaller scale, where we only have to go back a couple of weeks to find an example. Economist John Cochrane made a logical mistake in arguing that property taxes are progressive, which is to say that people who earn higher annual incomes pay higher property taxes than people who earn lower annual incomes.

In search of support for his argument, he requested pointers to data indicating property taxes paid by income level from his readers, who responded with results that directly contradicted his argument. How he handled the contradiction demonstrates considerable integrity.

Every now again in writing a blog one puts down an idea that is not only wrong, but pretty obviously wrong if one had stopped to think about 10 minutes about it. So it is with the idea I floated on my last post that property taxes are progressive.
Morris Davis sends along the following data from the current population survey.
No, Martha (John) property taxes are not progressive, and they're not even flat, and not even in California where there is such a thing as a $10 million dollar house. (In other states you might be pressed to spend that much money even if you could.) People with lower incomes spend a larger fraction of income on housing, and so pay more property taxes as a function of income. Mo says this fact is not commonly recognized when assessing the progressivity of taxes.

Not only is Cochrane providing insight into the error in his thinking, his transparency in addressing why it was incorrect is helping to advance the general knowledge of his readers.

In both these cases, we have examples of problems that could have been simply swept under a rug and virtually ignored with nobody being the wiser, but where the ethical standards of the people involved wouldn't let them do that. Even in making mistakes while addressing the mistakes they made or discovered, they made the problems known as they worked to resolve them, and because they did so, we can have greater confidence in trusting the overall quality of their work.

In the real world where people value honesty, admitting or acknowledging errors is no penalty. In fact, there are solid examples where scientists have retracted papers because of errors they made that, ultimately, had zero impact on their careers. Which in some cases, involved going on to be awarded Nobel prizes in their fields.

Retracting a paper is supposed to be a kiss of death to a career in science, right? Not if you think that winning a Nobel Prize is a mark of achievement, which pretty much everyone does.
Just ask Michael Rosbash, who shared the 2017 Nobel Prize in physiology or medicine for his work on circadian rhythms, aka the body's internal clock. Rosbash, of Brandeis University and the Howard Hughes Medical Institute, retracted a paper in 2016 because the results couldn't be replicated. The researcher who couldn't replicate them? Michael Young, who shared the 2017 Nobel with Rosbash.
This wasn't a first. Harvard's Jack Szostak retracted a paper in 2009. Months later, he got that early morning call from the Nobel committee for his work. And he hasn't been afraid to correct the record since, either. In 2016, Szostak and his colleagues published a paper in Nature Chemistry that offered potentially breakthrough clues for how RNA might have preceded DNA as the key chemical of life on Earth - a possibility that has captivated and frustrated biologists for half a century. But when Tivoli Olsen, a researcher in Szostak's lab, repeated the experiments last year, she couldn't get the same results. The scientists had made a mistake interpreting their initial data. Once that realization settled in, they retracted the paper - a turn of events Szostak described as "definitely embarrassing."
What isn’t absurd is the idea that admitting mistakes shouldn’t be an indelible mark of Cain that kills your career.

Indeed it shouldn't. And for honest people with high standards of ethical integrity and a willingness to be transparent about the mistakes that they have made or that have occurred on their watch, it isn't.