HIGGINSON, Circuit Judge:

Humberto Homero Duron-Caldera appeals his conviction for illegal reentry. On appeal, he argues that the district court's admission of his grandmother's affidavit violated his Confrontation Clause rights. For the reasons articulated below, we VACATE Duron-Caldera's conviction and REMAND for further proceedings consistent with this opinion.

I.

On March 16, 2011, a federal grand jury indicted Duron-Caldera with one count of illegal reentry after deportation, in violation of 8 U.S.C. § 1326(a). In order to convict him of this offense, the government was required to prove that he was an alien. See 8 U.S.C. § 1326(a). Prior to trial, defense counsel indicated that Duron-Caldera's defense theory would be that the government could not prove beyond a reasonable doubt that he did not derive citizenship through his United States — citizen mother, Maria Rosa Caldera de Duron ("Maria Caldera"). Under 8 U.S.C. § 1401, Duron-Caldera could derive citizenship through Maria Caldera if, prior to his birth in 1962, she had been physically present in the United States for ten years, at least five of which were after she reached the age of fourteen. See 8 U.S.C. § 1401(a)(7) (1952) (current version at 8 U.S.C. § 1401(g)).1

To prove Duron-Caldera's alienage, the government sought to introduce a sworn affidavit of his maternal grandmother, Francisca Serrato de Caldera ("Serrato Affidavit"), stating that Maria Caldera lived in the United States from September 1960 until April 1961. Serrato swore to the affidavit in 1968 in connection with an investigation into document fraud, including the alleged filing of fraudulent birth certificates by Duron-Caldera's parents and Serrato. The affidavit is on an immigration form signed by Serrato, an immigration officer, and a witness.

In the affidavit, Serrato recounts the dates and locations of her children's and grandchildren's births; accuses a midwife named Guadalupe San Miguel of falsely registering the births of four of Serrato's grandchildren in Texas; accuses her son and two of her sons-in-law of arranging these false registrations; refutes an allegation of wrongdoing; and denies any involvement in arranging the false registrations.2

Duron-Caldera moved in limine to exclude the Serrato Affidavit. He argued that admission of the affidavit would violate his Confrontation Clause rights because the affidavit is testimonial hearsay, Serrato is deceased and therefore cannot testify at trial, and he had no prior opportunity to cross-examine Serrato. At a pretrial conference, the district court took the motion under advisement.

At trial, Duron-Caldera again objected to admission of the Serrato Affidavit. During a bench conference, the government candidly acknowledged that Serrato swore to and made the affidavit in connection with a document fraud investigation and that Guadalupe San Miguel was criminally prosecuted and convicted pursuant to that investigation. After hearing arguments from both sides, the district court overruled the objection on the ground that the affidavit is nontestimonial because it was not created to accuse Duron-Caldera in his illegal reentry trial.

The government introduced the Serrato Affidavit through Maria Flores, a Citizenship and Immigration Services officer. Flores testified that she found the Serrato Affidavit in the alien files ("A-Files") of Duron-Caldera's parents. Although she testified that immigration affidavits are "kept in the normal course of business of defendant affidavits," she did not testify to, or express any knowledge of, the circumstances surrounding the creation and narrative contents of the Serrato Affidavit in particular.

The government introduced a number of other documents to prove Duron-Caldera's alienage, including the denial of his United States citizenship application; his admission of Mexican citizenship at the time of arrest; and Maria Caldera's citizenship application, in which she states that she arrived in the United States in September 1960, only two years before Duron-Caldera's birth.

The defense, in turn, introduced documents to show that Maria Caldera may have met the residency requirement. The defense introduced a record of Maria Caldera's attendance at a Texas school from 1947 through 1948; her brother's 1947 Texas birth certificate; a certificate of her sister's 1952 baptism in a Texas church; and an affidavit stating that her father worked in Texas from 1950 until 1960.

In its closing argument, the government argued that the Serrato Affidavit proved that Duron-Caldera did not derive citizenship through his mother. The district court then gave final instructions and submitted the case to the jury. After deliberating ninety minutes, the jury indicated it was deadlocked. The district court informed the parties of the deadlock and, pursuant to their joint request, instructed the jury to continue deliberations. After deliberating another eighty minutes, the jury returned a guilty verdict. The court sentenced Duron-Caldera to ninety-two months imprisonment and three years of supervised release. He filed a timely notice of appeal.

II.

On appeal, Duron-Caldera claims that the district court's admission of the Serrato Affidavit violated his Sixth Amendment right to confront Serrato. The government responds that admission of the affidavit was proper and that, alternatively, any error in admitting the affidavit was harmless. This court reviews Duron-Caldera's Confrontation Clause objection de novo, subject to harmless error analysis. See United States v. Polidore, 690 F.3d 705, 710 (5th Cir.2012).

A.

The Confrontation Clause of the Sixth Amendment provides that, "[i]n all criminal prosecutions, the accused shall enjoy the right ... to be confronted with the witnesses against him." U.S. Const. amend. VI. In Crawford v. Washington, 541 U.S. 36, 53-54, 124 S.Ct. 1354, 158 L.Ed.2d 177 (2004), the Supreme Court held that the Clause bars the admission of "testimonial statements of a witness who did not appear at trial unless he was unavailable to testify, and the defendant had [] a prior opportunity for cross-examination." In this case, Serrato was unavailable to testify and Duron-Caldera did not have a prior opportunity for cross-examination. Accordingly, Duron-Caldera's Confrontation Clause challenge turns on whether the Serrato Affidavit is testimonial.

In Crawford, the Court defined "testimony" as "[a] solemn declaration or affirmation made for the purpose of establishing or proving some fact." Id. at 51, 124 S.Ct. 1354. The Court then described the "core class of testimonial statements" to include:

Id. at 51-52, 124 S.Ct. 1354 (internal quotation marks and citations omitted).

Following Crawford, the Court adopted the "primary purpose" test for determining the testimonial nature of statements. See Davis v. Washington, 547 U.S. 813, 822, 126 S.Ct. 2266, 165 L.Ed.2d 224 (2006). Under this test, a statement is testimonial if its "primary purpose ... is to establish or prove past events potentially relevant to later criminal prosecution." Id.; see also Bullcoming v. New Mexico, ___ U.S. ___, 131 S.Ct. 2705, 2714 n. 6, 180 L.Ed.2d 610 (2011).

Significantly, "the government bears the burden of defeating [a] properly raised Confrontation Clause objection by establishing that its evidence is nontestimonial." United States v. Jackson, 636 F.3d 687, 695 & n. 4 (5th Cir.2011).

B.

The government has not met its burden in this case to prove that the Serrato Affidavit is nontestimonial. In describing the "core class of testimonial statements," the Court in Crawford mentions affidavits twice. Melendez-Diaz v. Massachusetts, 557 U.S. 305, 310, 129 S.Ct. 2527, 174 L.Ed.2d 314 (2009). Indeed, the Serrato Affidavit is "quite plainly" an affidavit. See id. (defining "affidavits" as "declaration[s] of facts written down and sworn to by the declarant before an officer authorized to administer oaths" (quoting Black's Law Dictionary 62 (8th ed.2004))). Officer Flores identified it as "an affidavit by a witness." It is entitled "Record of Sworn Statement in Affidavit Form; Affidavit — Witness," sworn by the declarant, signed by an officer authorized to administer oaths, and witnessed by another. In the affidavit, Serrato recounts the number of years Duron-Caldera's mother lived in the United States prior to his birth. This is "the precise testimony [she] would be expected to provide if called at trial." Id. The affidavit is "functionally identical to live, in-court testimony, doing `precisely what a witness does on direct examination.'" Id. at 310-11, 129 S.Ct. 2527 (quoting Davis, 547 U.S. at 830, 126 S.Ct. 2266).

The government contends that the Serrato Affidavit is nontestimonial because it was created for the primary purpose of providing evidence for immigration, rather than criminal, proceedings.3 The government, however, has not met its burden to prove this assertion. The record reveals little regarding the circumstances surrounding the creation of the affidavit. Serrato is deceased. Others present for its creation — the immigration officer, interpreter, and witness — did not testify at the pretrial conference or at trial. If the government has files regarding the fraud investigation, it did not share them with the court.

The evidence we do have is inconclusive. The affidavit is on an immigration form signed by an immigration officer. Yet, documents prepared by immigration officers on immigration forms can be testimonial if created for use at a later criminal trial. See, e.g., United States v. Martinez-Rios, 595 F.3d 581, 586 (5th Cir.2010) (finding that a Certificate of Nonexistence of Record from the defendant's A-File was testimonial). There is evidence in this case to suggest that the affidavit was created for use at a later criminal trial, including: (1) the government's concession that the affidavit was taken as part of a document fraud investigation that resulted in a criminal prosecution and conviction, and (2) the affidavit itself, in which Serrato exculpates herself and inculpates four others in the fraud.

Based on our review of the record, we conclude that the government has failed to establish that the Serrato Affidavit was not created for the primary purpose of providing evidence for a later criminal trial. See Jackson, 636 F.3d at 696-97. Because the government has not met its burden to prove that the affidavit is nontestimonial, the district court erred in admitting the affidavit.

C.

The government also contends that the Serrato Affidavit is nontestimonial because it was not made to "accuse" Duron-Caldera of illegal reentry. The government correctly observes that when the affidavit was created forty years ago, neither Serrato nor law enforcement could have anticipated that the contents of the affidavit would inculpate Duron-Caldera. The government cites Williams v. Illinois, ___ U.S. ___, 132 S.Ct. 2221, 183 L.Ed.2d 89 (2012) in support of this argument. In Williams, four Justices, referred to here as the plurality, expressed support for a primary purpose test that would find testimonial only statements "having the primary purpose of accusing a targeted individual of engaging in criminal conduct." Id. at 2242-43.

We decline to adopt the government's proposed "accusation" test for a number of reasons. First, in Williams, five Justices expressly rejected this test. See id. at 2262 (Thomas, J., concurring) ("The [plurality's] new primary purpose test ... lacks any grounding in constitutional text, in history, or in logic."); id. at 2273 (Kagan, J., dissenting) (writing for Justices Scalia, Ginsburg, and Sotomayor) ("Where [the plurality's] test comes from is anyone's guess. Justice Thomas rightly shows that it derives neither from the text nor the history of the Confrontation Clause."). For this same reason, the plurality's test in Williams would not be controlling under Marks v. United States, 430 U.S. 188, 193, 97 S.Ct. 990, 51 L.Ed.2d 260 (1977). See United States v. James, 712 F.3d 79, 95 (2d Cir.2013).4

Second, along with these five Justices, we do not find support for the proposed test in the text of the Confrontation Clause. The Sixth Amendment provides a criminal defendant ("the accused") with the right "to be confronted with the witnesses against him." U.S. Const. amend. VI (emphasis added). The textual juxtaposition, therefore, is not between "the accused" and his "accuser"; it is between "the accused" and "the witnesses against him." To the extent Serrato was a witness (discussed earlier), she "certainly provided testimony against petitioner, proving one fact necessary for his conviction" — his alienage. See Melendez-Diaz, 557 U.S. at 313, 129 S.Ct. 2527. "The text of the [Sixth] Amendment contemplates two classes of witnesses — those against the defendant and those in his favor.... [T]here is not a third category of witnesses, helpful to the prosecution, but somehow immune from confrontation." Id. at 313-14, 129 S.Ct. 2527.

Third, we do not find support for the proposed test in Confrontation Clause precedent. Under Supreme Court precedent, a statement is testimonial if made for the primary purpose of establishing "past events potentially relevant to later criminal prosecution." Davis, 547 U.S. at 822, 126 S.Ct. 2266; see also Bullcoming, 131 S.Ct. at 2714 n. 6, 2716-17; Michigan v. Bryant, ___ U.S. ___, 131 S.Ct. 1143, 1155-57, 1165, 179 L.Ed.2d 93 (2011); Melendez-Diaz, 557 U.S. at 310-11, 129 S.Ct. 2527; Crawford, 541 U.S. at 51-52, 124 S.Ct. 1354. "None of our cases has ever suggested that, in addition, the statement must be meant to accuse a previously identified individual." Williams, 132 S.Ct. at 2274 (Kagan, J., dissenting); see also Kirby v. United States, 174 U.S. 47, 54-61, 19 S.Ct. 574, 43 L.Ed. 890 (1899).5 In Melendez-Diaz, we perceive that the Court rejected this requirement. Melendez-Diaz, 557 U.S. at 313, 129 S.Ct. 2527 (rejecting respondent's argument "that the analysts are not subject to confrontation because they are not `accusatory' witnesses, in that they do not directly accuse petitioner of wrongdoing"). Tracing back to Crawford, moreover, we discern that the Court has identified the concern of the Confrontation Clause as statements one "would reasonably expect to be used prosecutorially ... [or] at a later trial." Crawford, 541 U.S. at 51-52, 124 S.Ct. 1354 (emphasis added).

Finally, the proposed test relies on an overly-narrow view of the rationale behind confrontation. The test assumes that the Confrontation Clause is designed to protect only against "a defendant-related motive to behave dishonestly." See Williams, 132 S.Ct. at 2274 (Kagan, J., dissenting) (quoting Williams, 132 S.Ct. at 2250 (Breyer, J., concurring)). But confrontation protects against a wide range of witness reliability concerns beyond personal bias, such as perception, memory, narration, and sincerity. See id. at 2249 (Breyer, J., concurring) (citing 30 C. Wright & K. Graham, Federal Practice and Procedure § 6324, pp. 44-49 (1997)); see also Melendez-Diaz, 557 U.S. at 319, 129 S.Ct. 2527 ("Confrontation is designed to weed out not only the fraudulent analyst, but the incompetent one as well."). The Confrontation Clause commands only one method of testing these witness-related reliability concerns: "the crucible of cross-examination." Crawford, 541 U.S. at 61, 124 S.Ct. 1354.6

D.

We turn now to the question of whether the district court's error in admitting the Serrato Affidavit was harmless. A defendant deprived of the right to confront witnesses against him is entitled to a new trial unless the government proves beyond a reasonable doubt that the error was harmless; that is, that "there was [no] reasonable possibility that the evidence complained of might have contributed to the conviction." United States v. Alvarado-Valdez, 521 F.3d 337, 341 (5th Cir.2008) (quoting Chapman v. California, 386 U.S. 18, 24, 87 S.Ct. 824, 17 L.Ed.2d 705 (1967)). Relevant considerations include:

Id. (quoting United States v. Rodriguez-Martinez, 480 F.3d 303, 308 (5th Cir. 2007)).

A number of considerations weigh against a finding of harmlessness. This is not a case where cross-examination was permitted but improperly circumscribed. E.g., United States v. Jimenez, 464 F.3d 555, 563 (5th Cir.2006). Duron-Caldera had no opportunity to cross-examine Serrato and there was ample grist for cross-examination: Serrato made the statements more than forty years before trial, some of the statements were self-serving, and the affidavit does not indicate Serrato's basis of knowledge. These weaknesses, if probed on cross-examination, could have undermined the reliability of the affidavit and, in turn, affected the jury's verdict. See Delaware v. Van Arsdall, 475 U.S. 673, 684, 106 S.Ct. 1431, 89 L.Ed.2d 674 (1986); Jimenez, 464 F.3d at 563.

The Serrato Affidavit was also an important part of the prosecution's case. Duron-Caldera's derivative-citizenship theory was his sole defense, and the Serrato Affidavit was one of only two pieces of evidence the government used to prove that Maria Caldera did not meet the ten-year residency requirement. The jury's initial deadlock suggests that the jury seriously considered Duron-Caldera's derivative-citizenship defense.

Significantly, the government emphasized the affidavit in its closing argument. The prosecutor told the jury that "[t]he only way [Duron-Caldera] could have acquired citizenship would have been through his mother." The prosecutor then pointed to the "affidavit of the defendant's grandmother" as evidence "prov[ing] that did not occur." In light of the government's reliance on the affidavit in closing, "[w]e cannot see how the government can conclusively show that the tainted evidence did not contribute to the conviction." Alvarado-Valdez, 521 F.3d at 342-43; see also Jackson, 636 F.3d at 697.

The government contends that any confrontation error was harmless because the Serrato Affidavit was cumulative. It is true that Serrato's statement is corroborated by Maria Caldera's citizenship application, in which she claims she arrived in the United States in 1960. But for evidence to be considered cumulative in this context, "substantial evidence [must] support[] the same facts and inferences as those in the erroneously admitted evidence." United States v. Demmitt, 706 F.3d 665, 673 (5th Cir.2013). The Serrato Affidavit does not meet this standard for two reasons. First, Maria Caldera's citizenship application was the only other evidence corroborating Serrato's statement and it, too, was not subject to cross-examination. No trial witnesses testified to whether Maria Caldera met the ten-year residency requirement. Second, there was other evidence contradicting the Serrato Affidavit and Maria Caldera's application, including school records, birth certificates, and baptismal records indicating that Maria Caldera and her family were present in the United States in the 1940s and 1950s. Based on our review of the record, we conclude that the government did not prove that the district court's error was harmless beyond a reasonable doubt.

III.

For the foregoing reasons, we VACATE the judgment of conviction and REMAND for further proceedings consistent with this opinion.