FOR PUBLICATION
 UNITED STATES COURT OF APPEALS
      FOR THE NINTH CIRCUIT

UNITED STATES OF AMERICA,                 No. 06-30258
                Plaintiff-Appellee,
               v.                           D.C. No.
                                          CR-05-00079-EJL
RUSSELL LAROY HOLLAND,
                                             OPINION
             Defendant-Appellant.
                                      
       Appeal from the United States District Court
                 for the District of Idaho
        Edward J. Lodge, District Judge, Presiding

                  Argued and Submitted
            February 8, 2007—Portland, Oregon

                 Filed September 4, 2007

   Before: David R. Thompson, Andrew J. Kleinfeld, and
               Jay S. Bybee, Circuit Judges.

                 Opinion by Judge Bybee




                           11411
11414             UNITED STATES v. HOLLAND


                        COUNSEL

Philip Gordon, Boise, Idaho, for appellant Russell Laroy Hol-
land.

Alan G. Burrow & George W. Breitsameter, Boise, Idaho, for
appellee United States of America.


                         OPINION

BYBEE, Circuit Judge:

   Russell Laroy Holland appeals his conviction and sentence
for mailing threatening communications and threatening the
                  UNITED STATES v. HOLLAND              11415
President of the United States. He maintains that the district
court judge who imposed the sentence should have recused
himself after Holland obtained the judge’s home telephone
number and left at least one threatening message prior to his
sentencing. We hold that the district judge reasonably con-
strued Holland’s threatening phone message as an attempt to
manipulate the court system which did not warrant his sua
sponte recusal. Accordingly, we affirm the judgment.

                              I

   Russell Laroy Holland pled guilty to one count each under
18 U.S.C. § 876 and 18 U.S.C. § 871 for mailing threatening
communications and threatening the President of the United
States. He agreed to plead guilty in exchange for the Govern-
ment dismissing the other four counts of the indictment. On
April 13, 2006, Appellant was sentenced to seventy-eight (78)
months incarceration followed by three (3) years of super-
vised release, to begin running after Holland served his cur-
rent state sentence. Holland is projected to be released from
Federal custody sometime during the year 2012, when he will
be 51 years old.

   Holland has a lengthy criminal record and has been incar-
cerated for most of his adult life for crimes ranging from
armed robbery, petty theft, escape from prison, assault by a
prisoner, grand theft, robbery, prisoner possession of a
weapon, assault with a deadly weapon, threats against state
officials, and injury to jails. At some point during the pro-
ceedings in this case, Holland obtained the sentencing judge’s
home telephone number, which he called, leaving more than
one threatening message on an answering machine. Before
sentencing, the district court revealed that he had received
these threatening messages from Holland. The district court,
however, dismissed them as attempts to “manipulate the sys-
tem.” Specifically, the judge remarked:

    For the record, too, the Court is also aware that Mr.
    Holland did, in fact, leave voice messages at the
11416                 UNITED STATES v. HOLLAND
      Court’s personal residence that to some people could
      be construed as threatening. However, in my judg-
      ment, Mr. Holland’s history clearly demonstrates
      that these are attempts to manipulate the criminal
      justice system rather than threats as such and the
      Court is just simply not going to allow Mr. Holland
      to manipulate the system. So the Court has chosen to
      go forward with this sentencing at this time.

Later in the proceeding, the judge observed that Holland had
a history of violent and assaultive crimes and “if he is given
the opportunity, he has the ability to carry out his threats . . . .
[I]t is clearly important . . . that the Court impose[ ] a sentence
for the protection of society.” Holland did not object to the
sentencing judge’s decision to proceed with the hearing or
request that the judge recuse himself.

                                     II

   We are confronted with a narrow question: When does a
judge have an obligation under 28 U.S.C. § 455 to recuse
himself sua sponte in response to threats made against him,
his family members or associates? Because the issue was not
raised before the trial court, we review for plain error. Jones
v. United States, 

, 388 (1999); United States v.
Olano, 

, 736 (1993); United States v. Ortiz, 

, 1278 (9th Cir. 2004).1 In Noli v. Comm’r of Inter-
nal Revenue, 

, 1527 (9th Cir. 1988), we held
that “[f]ailure to move for recusal at the trial level . . . does
not preclude raising on appeal the issue of recusal under
§ 455.” Id. “ ‘Nonetheless, if no motion is made to the [trial
court] judge . . . a party will bear a greater burden on appeal
  1
   The government’s argument that Holland’s motion for recusal must be
timely is irrelevant. The cases the government cites for this proposition all
involved motions made to the district court. No such motion was made
here. We, therefore, consider the issue for the first time on appeal and
review for plain error.
                     UNITED STATES v. HOLLAND                     11417
in demonstrating that the judge . . . [erred] in failing to grant
recusal under section 455.’ ” Id (quoting United States v.
Sibla, 

, 868 (9th Cir. 1980)); see also Pau v.
Yosemite Park and Curry Co., 

, 885 (9th Cir.
1991).

                                   A

   [1] We begin with the general proposition that, in the
absence of a legitimate reason to recuse himself, a judge has
a duty to sit in judgment in all cases coming before him. See
Laird v. Tatum, 

, 837 (1972); Sensley v. Albrit-
ton, 

, 598-99 (5th Cir. 2004); United States v.
Snyder, 

, 46 (1st Cir. 2000); Nichols v. Alley, 

, 351 (10th Cir. 1995). The duty inheres in the “judi-
cial Power” with which we are vested. See U.S. CONST. art III,
§ 1. It is reflected in our oath, by which we have obligated
ourselves to “faithfully and impartially discharge and perform
[our] duties” and to “administer justice without respect to per-
sons, and do equal right to the poor and to the rich.” 28 U.S.C.
§ 453. Without a duty to sit, we could recuse ourselves for
any reason or no reason at all; we could pick and chose our
cases, abandoning those that we find difficult, distasteful,
inconvenient or just plain boring. Our mythic Justice, repre-
sented by a blindfolded figure wielding a balance and a
sword, hears all cases coming before her, giving no
preference—whether in priority or result—to the station or
economic status of such persons. We are duty-bound to sit in
all cases—not just the ones we would like to hear—except
those in which our “impartiality might reasonably be ques-
tioned.” 28 U.S.C. § 455(a); see also id. § 455(b) (enumerat-
ing circumstances requiring recusal).

   [2] We also recognize that the security of the nation’s judi-
ciary is a serious concern. We are, unfortunately, reminded
from time to time that threats against the judiciary have been
carried out.2 We stress that, despite the duty to sit, if a judge
  2
   Despite security measures, threats against federal judges have taken a
deadly turn. In 1979, Judge John H. Wood Jr., of the Western District of
11418                 UNITED STATES v. HOLLAND
feels that his personal safety or the safety of his family is in
danger, he may always recuse himself sua sponte from a mat-
ter. Recusal in such situations is left to the discretion of the
threatened judge, who is in the best position to evaluate the
advisability of recusal; we generally do not review a judge’s
decision to recuse himself.

   [3] The question before us, however, is different. Here we
must decide when a judge must recuse himself sua sponte in
response to threats even if he would prefer to continue his
work on the case. Here, Holland contends that under 28
U.S.C. § 455, the district judge should have recused himself
sua sponte after receiving his threatening phone calls.

  [4] Section 455(a) of Title 28 reads: “Any justice, judge or
magistrate judge of the United States shall disqualify himself

Texas, was shot to death outside his home by a hit man hired to prevent
him from presiding over a narcotics trial. In 1988 Judge Richard J.
Daronco, of the Southern District of New York, was shot outside his home
by a retired police officer after the judge dismissed a sexual discrimination
suit brought by the officer’s daughter. Judge Robert S. Vance, of the Elev-
enth Circuit, was killed at his home by a pipe bomb mailed by an inmate
angry that his prior conviction was not overturned. In 2005, Chicago Dis-
trict Court Judge Joan Lefkow’s husband and mother were brutally mur-
dered by a man whom the judge had ruled against in court. See Rick
Lyman, “Focus on Safety for Judges Outside the Courtroom,” N.Y. TIMES,
Mar. 11, 2005, at A18; John Bebow & John Keilman, “Man commits sui-
cide, claims responsibility for Lefkow murders,” CHICAGO TRIBUNE, Mar.
11, 2005, at A1. The wife of Judge Charles Brieant of the Southern Dis-
trict of New York was rushed to the hospital in 1987 after eating poisoned
chocolates mailed by a former university professor jailed by Brieant for
manufacturing drugs in his NYU laboratory. Reuters, “FBI Says Ex-Prof
Sent Judge ‘Sweet Revenge,’ ” PHILADELPHIA DAILY NEWS, Feb. 21, 1987,
at 10. We are mindful that our state counterparts are also at risk.
   At the time of the Lefkow murders, the U.S. Marshal’s service, tasked
with providing security for the federal judiciary, logged roughly 700
threats or “inappropriate communications” against judges each year.
Amanda Paulson & Brad Knickerbocker, “Chicago Murders Spotlight
Risk to Judges,” CHRISTIAN SCI. MONITOR, Mar. 3, 2005, at 3.
                       UNITED STATES v. HOLLAND                        11419
in any proceeding in which his impartiality might reasonably
be questioned.” In 1974 Congress amended the statute to
replace the subjective test of the previous version of the stat-
ute with an objective test based on public perception.3 The
Supreme Court discussed this change in Liljeberg v. Health
Servs. Acquisition Corp., explaining that, “The goal of section
455(a) is to avoid even the appearance of partiality. If it
would appear to a reasonable person that a judge has knowl-
edge of facts that would give him an interest in the litigation
then an appearance of partiality is created even though no
actual partiality exists.” 

, 860 (1988) (internal
quotation marks omitted). We have restated § 455(a) and ask
“whether a reasonable person with knowledge of all the facts
would conclude that the judge’s impartiality might reasonably
be questioned.” Clemens v. U.S. Dist. Ct., 

,
1178 (9th Cir. 2005) (internal quotation marks and citation
omitted); cf. United States v. Cooley, 

, 993 (10th
Cir. 1993) (articulating a similar standard). “Section 455(a)
asks whether a reasonable person perceives a significant risk
that the judge will resolve the case on a basis other than the
merits.” In re Mason, 

, 385 (7th Cir. 1990). The
“reasonable person” is not someone who is “hypersensitive or
unduly suspicious,” but rather is a “well-informed, thoughtful
observer.” Id. at 386. The standard “must not be so broadly
construed that it becomes, in effect, presumptive, so that
recusal is mandated upon the merest unsubstantiated sugges-
tion of personal bias or prejudice.” Cooley, 1 F.3d at 993.
  3
    The prior version of the statute read, in part, “Any justice or judge of
the United States shall disqualify himself in any case in which he has a
substantial interest . . . or is so related or connected with any party or his
attorney as to render it improper, in his opinion, for him to sit on the trial,
appeal, or other proceeding therein.” 28 U.S.C. § 455 (1970) (emphasis
added). The 1974 amendment removed the subjective “in his opinion” and
replaced it with an “objective” test designed to “promote public confi-
dence in the integrity of the judicial process.” Liljeberg v. Health Servs.
Acquisition Corp., 

, 858 n.7 (1988); see also S. REP. NO. 93-
419, at 5.
11420                 UNITED STATES v. HOLLAND
   [5] Disqualification under § 455(a) is necessarily fact-
driven and may turn on subtleties in the particular case. Con-
sequently, “the analysis of a particular section 455(a) claim
must be guided, not by comparison to similar situations
addressed by prior jurisprudence, but rather by an indepen-
dent examination of the unique facts and circumstances of the
particular claim at issue.” United States v. Bremers, 

, 226 (5th Cir. 1999); see also Clemens, 428 F.3d at 1178.
In general, the conscientious judge should also bear in mind
that § 455 is limited by the “extrajudicial source” factor which
generally requires as the basis for recusal something other
than rulings, opinions formed or statements made by the judge
during the course of trial. Liteky v. United States, 

, 554-56 (1994). Put differently, the judge’s conduct dur-
ing the proceedings should not, except in the “rarest of circum-
stances”4 form the sole basis for recusal under § 455(a). Id. at
555.5
  4
     We note that these “rarest of circumstances” occasionally arise where
events in the courtroom so “embroil [a judge] in controversy . . . that there
was such a likelihood of bias or an appearance of bias that the judge was
unable to hold the balance between vindicating the interests of the court
and the interests of the accused” and a new judge should assume control
of the case. Taylor v. Hayes, 

, 501 (1974) (internal quotation
marks omitted); see also United States v. Dellinger, 

 (7th Cir.
1972) (trial of the “Chicago Seven”).
   5
     We, along with our sister circuits, have identified various matters
which will not ordinarily require recusal under § 455: (1) “rumor, specula-
tion, beliefs . . . and similar non-factual matters;” (2) “the mere fact that
a judge has previously expressed an opinion on a point of law;” (3) “prior
rulings in the proceeding;” (4) “mere familiarity with the defendant(s) or
the type of charge;” (5) “baseless personal attacks on or suits against the
judge by a party;” (6) “reporters’ personal opinions or characterizations;”
and (7) “threats or other attempts to intimidate the judge.” Cooley, 1 F.3d
at 993-94; see also United States v. Burger, 

, 1069 (10th
Cir. 1992); Leaman v. Ohio Dep’t of Mental Retardation & Dev. Disabili-
ties, 

, 949 n.1 (6th Cir. 1987); United States v. Haldeman,

, 134 n.302 (D.C. Cir. 1976); United States v. Studley, 

, 940 (9th Cir. 1986); United States v. Greenough, 

,
1558 (11th Cir. 1986).
                  UNITED STATES v. HOLLAND               11421
                              B

   [6] Applying these general principles to situations where
the judge receives a threat, suggests a three-step process for
evaluating whether recusal is required under § 455. First, the
judge must evaluate the threat itself to determine how much
risk there is that it may be carried out and how much harm
there would be if it were. In deciding whether recusal is
appropriate, the judge may wish to consider the following fac-
tors:

    (1) The defendant’s capacity to carry out the threat.
    Has the defendant taken concrete steps to carry out
    the threat? Does the defendant have a history of vio-
    lence or has he previously been successful in carry-
    ing out other threats? Is he a member of a gang or
    does he have accomplices or contacts who could
    carry out the threat on his behalf?

    (2) The defendant’s demeanor and the context of the
    threat. Was the threat made in a fit of passion or
    intended as a joke? Does the judge believe that the
    defendant is serious in carrying out the threat? Does
    the judge have any prior dealings with the defendant
    that make the threat more or less likely to be carried
    out?

    (3) The perceived purpose of the threat. Was the
    threat made in open court or did the judge become
    aware of the threat only through the fortuity of a law
    enforcement investigation? Does the judge believe it
    is an attempt to force recusal and manipulate the
    judicial system?

The final factor is, perhaps, the most important. Not every
threat made against a judge should force recusal. If so, defen-
dants could readily manipulate the system, threatening every
jurist assigned on the “wheel” until the defendant gets a judge
11422                  UNITED STATES v. HOLLAND
he preferred. Also, the defendant could force delays, perhaps
making the cases against him more difficult to try, perhaps
putting witnesses at greater risk. Such blatant manipulation
would subvert our processes, undermine our notions of fair
play and justice, and damage the public’s perception of the judi-
ciary.6 We agree with the Eighth Circuit that “recusal is not
automatic on the mere basis of the judge’s knowledge of the
threat.” United States v. Gamboa, 

, 817 (8th Cir.
2006).

   [7] The second step is for the judge to determine whether
he can be truly impartial when trying case. If he feels he can-
not hear the case without bias, on account of the threat, then
the judge has a duty to recuse himself irrespective of how it
looks to the public. Section 455 of Title 28 requires recusal
where the judge “has a personal bias or prejudice concerning
a party.” 28 U.S.C. § 455(b)(1); see also 28 U.S.C. § 144
(addressing recusal upon motion by a litigant); CODE OF CON-
DUCT FOR UNITED STATES JUDGES CANON 3 (2000) (imposing
upon judges a duty to recuse themselves where they are either
impartial or when their impartiality may reasonably be ques-
tioned). This step is highly personal in nature and requires
each judge in such a situation to set aside emotion and
thoughtfully examine his ability to impartially “administer
justice without respect to persons.” 28 U.S.C. § 453. If he
  6
    The danger in a case where the judge has himself become the target is
twofold. First, there is the risk that the judge will accede to the pressure
posed by the threats. In that instance the judge is hostage to the threats and
goes easy on the defendant in order to defuse the threats and protect him-
self and his family. But there is a second consideration that a conscien-
tious judge should be aware of as well, namely that the judge will favor
the government in a criminal case as a means of protecting himself or his
family by imposing a tougher sentence on the person making the threats.
Neither scenario is acceptable; we are both judges and people and cannot
put our personal lives on the hanger from which we take our robes. There
are situations in which it is too much to expect a judge to dispense justice
faithfully and impartially, without regard for his own physical safety and
the safety of his family.
                   UNITED STATES v. HOLLAND                11423
feels there is a risk of prejudice, it is incumbent on him to
recuse himself from the case; failure to do so would amount
to an abdication of duty and be in clear derogation of the sol-
emn promise he made when he took his oath of office.

   [8] The third step in evaluating whether recusal is required
is to apply the “objective” standard articulated in Liljeberg,
486 U.S. at 860. The standard requires recusal if a reasonable
third-party observer would perceive that there is a “significant
risk” that the judge will be influenced by the threat and
resolve the case on a basis other than the merits. The reason-
able third-party observer is not a “partly informed man-in-the-
street,” but rather someone who “understand[s] all the rele-
vant facts” and has examined the record and law. LoCascio v.
United States, 

, 496 (2d Cir. 2007); see also
Clemens, 428 F.3d at 1178 (“The reasonable person in this
context means a well-informed, thoughtful observer, as
opposed to a hypersensitive or unduly suspicious person.”
(internal quotation marks and citation omitted)); but see In re
Nettles, 

, 1002 (7th Cir. 2005) (“We must bear
in mind that these outside observers are less inclined to credit
judges’ impartiality and mental discipline than the judiciary
itself will be.” (internal quotation marks and citation omit-
ted)). The “objective” standard is a check to avoid even the
“appearance of partiality,” Liljeberg, 486 U.S. at 860, and
insure that the judge’s decision is reasonable to an informed
observer.

   [9] Although the Supreme Court has explained the § 455
standard in objective terms, a threat to the judge’s person or
family necessarily involves the judge’s personal sentiment.
The judge must take into account his own perception of the
situation and personal feelings and ask himself if there is a
significant risk that the threat will influence his decision. Put
differently: Would a reasonable person understanding the
judge’s perspective accept the judge’s decision to recuse (or
not to recuse) himself? If the judge cannot, in good faith,
answer “yes,” then he should revisit his analysis under the
11424              UNITED STATES v. HOLLAND
first two steps. If it is a close case, the balance tips in favor
of recusal. United States v. Dandy, 

, 1349 (6th
Cir. 1993).

   [10] These considerations are, necessarily, deeply personal
to the judge. Only the judge himself, viewing the threats in
context, can differentiate between threats that the author
intended to be taken seriously and those made to anger or to
manipulate the system. And, only the judge can determine
whether he took them seriously or dismissed them. Although
§ 455 requires an objective standard on recusals, only the
judge—who is the sole person in possession of all the relevant
facts—must decide whether a reasonable person should be
concerned with risk that the threats will taint the proceedings
or create a perception of partiality.

                               C

   [11] Turning to the facts of the case at hand, it is apparent
to us that the district judge did not plainly err when he did not
recuse himself sua sponte after receiving Holland’s threaten-
ing phone message. The precise nature of the threat is not
clear from the record, but the district judge described it as a
message “that to some people could be construed as threaten-
ing.” The district court, however, did not consider the threats
or Holland’s capacity to carry them out serious enough to
refer the incident to the FBI, nor did he request additional
security from the U.S. Marshal’s service.

   Additionally, the judge carefully considered Holland’s
extensive (and curious) history of making threats. Holland
was before the district court to plead guilty of violating 18
U.S.C. § 871, making “knowingly and willful” threats against
the President of the United States. Section 871 does not
require proof of the defendant’s capacity, or evidence that the
defendant has taken any affirmative steps, to carry out such
threats. Holland’s current legal troubles also involved mailing
threatening letters to: (1) a state court judge (he later mailed
                      UNITED STATES v. HOLLAND                     11425
another letter apologizing for the threats); (2) to the prosecu-
tor involved in a prior criminal proceeding; and (3) the Presi-
dent of the United States in violation of 18 U.S.C. § 876.
When initially confronted with the letter to the President, Hol-
land told an Idaho Department of Corrections lieutenant
assisting the Secret Service that he did not want to remain in
state custody any longer and that he wanted to go to federal
prison instead. He thought he could get to federal prison by
committing a federal crime. Holland also threatened a jail
nurse in an apparent attempt to be transferred to the prison’s
medical facility and threatened his former defense lawyer in
an attempt to get him to mail a second letter from Holland
threatening the President. There is no evidence in the record
that Holland took any steps in furtherance of any of his
threats.

   [12] The district judge found that “these [threats] are
attempts to manipulate the criminal justice system.” This find-
ing is amply supported by the record, including Holland’s
behavior in the courtroom. At the beginning of the proceed-
ings, Holland blurted out, “The detectives told me yesterday
that I threatened you [the district judge] at home. Did I
threaten you at home?” Later, when given an opportunity to
address the court, Holland stated “I am not as bad as my
paperwork says . . . . There ain’t no way I am going to look
for a judge. There ain’t no damn way I am going to look for
a President of the United States or anybody . . . . I have never
hurt a victim.” He attempted to explain his threats as part of
the prison culture: “Maybe it’s because of the lifestyle, the
way prison is. Everybody threatens, ‘I am going to do this, I
am going to do that.’ It has got to the point where it is natural
to do that . . . . But as far as carrying out the threats, no. There
ain’t no way. Who would be stupid enough?” It was not until
well after Holland offered this explanation, and was subse-
quently removed from the courtroom by refusing to be quiet
during sentencing,7 that the judge addressed the issue of Hol-
land’s threatening phone messages.
  7
   The relevant trial transcript of this incident is as follows:
      THE COURT: Mr Holland, you can either listen to me or we can
11426                 UNITED STATES v. HOLLAND
   [13] Holland directs our attention to a comment the district
court made suggesting that Appellant’s threats should be
taken seriously. The court stated, “As I pointed out to him,
many of his crimes are violent and assaultive in nature. So
contrary to what he says, if he is given the opportunity, he has
the ability to carry out his threats.” Holland argues that this
statement suggests that the district court may have taken the
threats seriously, which would weigh in favor of recusal.8
However, there is compelling evidence that the district court
dealt with Holland in an even-handed manner. It considered
his motions, allowed Appellant to speak freely at sentencing,
and treated him in a courteous manner. When reviewing the
situation as a whole, we cannot conclude that a reasonable
person in possession of all the facts would determine that the
district court based its sentence on anything but the merits of
the case. Consequently, we find no plain error in the district

    have you removed, whichever way you want to do it. But right
    now I am ordering you not to say a word until I am finished.
    THE DEFENDANT: Let’s go. I want to be removed. I don’t kiss
    nobody’s ass.
      THE COURT: The record may show that the Marshal’s Service
      has escorted Mr. Holland out at his own request. He has been dis-
      ruptive and will not listen to the Court make its ruling in this
      case.
   8
     Appellant argues that the government’s decision to prosecute Appellant
for his threats only as his released date from state custody drew near is
evidence that it took his threats seriously and believed he had the ability
to carry them out. Whether or not the decision to prosecute was based in
part on the government’s belief that Appellant was a danger to society is
irrelevant to the issue of recusal. Prosecution under 18 U.S.C. §§ 876, 871,
does not require that the government prove that Appellant was likely to
carry out the threat; it only requires that a threat be made. Simply because
a person was prosecuted for making a threat does not necessarily mean
that a threat against a judge should be viewed as likely to be carried out.
If we were to hold otherwise, we would be creating a per se rule that any
threat made by a person convicted of a violent crime is de facto a threat
that should be taken seriously. We decline to adopt such a rule. The dis-
trict judge must evaluate each case on an individual basis.
                  UNITED STATES v. HOLLAND             11427
court’s decision not to recuse itself sua sponte under § 455.
The judgment is

  AFFIRMED.