A direct comparison of citations against the original sources.

Status: v2, 27 April 2026. Conducted by Colin and Claude. Documents verified: LEVIstrategia2030.pdf, Levi4_raportti_27032018-1.pdf, Kestävän matkailun suunnitelma 2022, State of Sustainable Tourism 2022, IPOL_STU_2018_629184_EN, LapinSuhdannekatsauskatsausvuoteen2024.pdf

Why this log exists

While drafting Page 2 of the public research site, a critique of the Levi 2030 strategy’s “Levikupla” concept rested on quotes and page references taken from NotebookLM-produced analyses of the source documents. Direct verification against the original Levi 2030 strategy revealed the page references were wrong and one of the framing claims was materially overstated.

This log is the systematic verification that should have been done before any drafting started. It exists to:

• Establish, citation by citation, what NotebookLM got right and wrong
• Identify which claims in the four NotebookLM analyses survive direct verification
• Document the pattern of errors so they can be guarded against in future
• Provide a corrections layer that future research and drafting can build on

This is not an attack on NotebookLM. It is an audit of a specific use of a specific tool, on a specific corpus, by a specific researcher (Colin, working in English from Finnish-language source material).

Headline findings

1. Page numbers are systematically unreliable. Across the verified citations, NotebookLM’s page numbers were wrong in roughly 90% of cases. Errors range from off-by-a-few-pages to citing pages that do not exist in the document.

2. Direct quotations are mostly accurate. Where NotebookLM gives a Finnish-language quotation, the quotation itself can usually be found verbatim somewhere in the cited document. The errors are in where it appears, not whether it appears.

3. Years and dates are sometimes wrong. During verification of the Lapin Suhdannekatsaus revenue figures, NotebookLM attributed the 2023 figure to 2022. The figure was real; the year was off by one.

4. Framing accuracy is mixed. For most claims, the framing reasonably reflects the document’s strategic intent. For a small but consequential subset, NotebookLM’s analytical interpretation overstates what the document actually commits to.

5. The most rhetorically punchy critique was the most overstated. The Levikupla critique — used as the most operationally specific commitment in the Levi 2030 strategy — turned out to be one target of five within one pillar of six in a strategic table, with the rhetorical “bubble” quote drawn from a brainstorm output rather than a strategic commitment.

6. NotebookLM occasionally produces unsourced numerical claims even when explicitly asked for sourced data. During the leakage estimate research, NotebookLM included an “industry-typical” commission range (15–25%) in its response while simultaneously stating that this figure was not in the source set.

7. NotebookLM’s negative findings are not the same as confirmed absences in the source corpus. The first two prompts in the leakage estimate research returned negative findings on Levi-specific data. The third prompt returned substantive Levi-specific data from documents already in the corpus.

8. NotebookLM may summarise sources less usefully than direct reading. The Lapin Suhdannekatsaus contains a full ten-year Levi revenue series plus parallel data for five other Lapland tourism centres. NotebookLM’s response surfaced one figure from this series. Direct reading produced ten times the data with one-tenth the ambiguity.

New verifications: leakage estimate research (April 2026)

V1: “Approximately 80% of Levi’s accommodation capacity is unregistered”

NotebookLM said: Quoted from Levi4_raportti_27032018-1.pdf, p. 61 and p. 80.

Direct check against the original document:

• The Finnish quote “Levin majoituskapasiteetista noin 80 % on rekisteröimätöntä” appears on page 24 of the Levi 4 report, in section 4.1.3 Levin matkailu tilastojen valossa (Levi tourism in the light of statistics).
• A second mention of “80%” on page 47 was checked. It is a different claim — it refers to safari companies’ international customer mix, not unregistered accommodation.

Verdict: Verified, with page reference correction. The headline claim is true and explicitly stated in the source. The correct citation is page 24, not pages 61 or 80.

V2: “24,500 beds in Levi, of which 4,000 are in hotels”

Direct check against the original document:

• Page 21 of Levi 4 contains a structured infographic stating: “Noin 24 500 vuodepaikkaa, joista 4000 hotelleissa” — approximately 24,500 beds, of which 4,000 in hotels.
• Page 15 contains a strategic capacity progression table: 16,000 beds in 2003, 24,500 in 2016, with targets of 30,000 by 2020 and 35,000 by 2025.
• Page 5 confirms the figure in narrative form.

Verdict: Verified. The 24,500 figure is the 2016 baseline. The hotel/non-hotel split is explicitly published: 4,000 hotel beds (16%), 20,500 non-hotel beds (84%).

V3: 2016 Levi visitor mix and origin breakdown

Direct check against the original document:

• Page 26 of Levi 4 contains a complete annual table titled “Kittilä (Levi) vuosittaiset yöpymiset, kpl”.
• The 2016 figures match exactly: Total 542,750 (Domestic 304,849, International 237,901). Domestic share: 56.2%, international: 43.8%.
• Country breakdowns match: UK 75,766, Germany 25,486, France 18,950, Norway 24,009, Russia 15,979.

Verdict: Verified. Citable to page 26 of Levi 4.

Caveat: The 2016 figures are now a decade old. The Russian market collapsed in 2022.

V4: “Kittilä tourism revenue €200m+ including indirect effects (2016)”

Direct check against the original document:

• Page 5 of Levi 4 states: “Vuonna 2016 Kittilän matkailutulo oli välillisine vaikutuksineen yli 200 miljoonaa euroa, joka on noin puolet Kittilän yritysten kokonaisliikevaihdosta”.
• Page 15 confirms the figure as “alueen nykyiseksi matkailutuloksi… arvioitu noin 200 MEUR”.

Verdict: Verified, with framing clarification. The figure is Kittilä-municipal-level and multiplier-inclusive. It is not the direct-revenue figure for Levi-specific companies.

V5: The Levi tourism revenue figure (€119m in 2023, €127m estimate 2024)

NotebookLM said: €118,990,634 in 2022, citing Lapin-Suhdannekatsaus-katsaus-vuoteen-2024.pdf, p. 53.

Direct check against the original document:

• The document is now in the project source folder (added 27 April 2026) and has been verified directly.
• NotebookLM was wrong about both the figure’s year and the page number. The €118,990,634 figure is the 2023 actual figure, not 2022. The 2022 actual figure in the same source is €107,448,525. The 2024 figure is an estimate of approximately €127m.
• The data appears on page 23 (Appendix 1/2 — Liikevaihdon rakennetiedot vuosilta 2014-2023 ja laskennallinen ennakkoarvio vuodelle 2024), not page 53 as NotebookLM claimed.
• Source authority: Statistics Finland customer-specific economic service (Tilastokeskus / Asiakaskohtainen suhdannepalvelu), as published by the Regional Council of Lapland.

Verdict: Verified, with year and page corrections. The figure is real, sourced to Statistics Finland’s primary customer-specific economic service, and published annually by the Regional Council of Lapland. The corrections to year and page reference are material.

Bonus finding: The Lapin Suhdannekatsaus contains the full Levi tourism revenue series from 2014 to 2024, plus parallel data for the other five Lapland tourism centres (Meri-Lappi, Pyhä-Luosto, Rovaniemi, Saariselkä, Ylläs). The complete series:

V6: 70% FIT / 30% packaged tour operator clientele for Lapland (2022)

NotebookLM said: Lapland regional strategy 2022 (Lapinliitto) p. 16 states 70% FIT / 30% tour operator split for European demand potential.

Direct check: The cited document, Lapinliitto-Matkailustrategia-2022-sivuina.pdf, is not in the project source folder available to this verification.

Verdict: Pending verification. The figure is plausible and consistent with documented European tourism patterns.

Action required before publication: Obtain the Lapland regional strategy document and verify directly.

V7: Rovaniemi Airbnb listings exceeding Barcelona per capita (2018)

NotebookLM said: Matkailu_tilannekuvaraportti_web.pdf p. 24 states Rovaniemi had three times Helsinki’s per-capita Airbnb listings, exceeding Barcelona’s per-capita rate.

Direct check: The cited document is not in the project source folder available to this verification.

Verdict: Pending verification. The framing is striking and rhetorically powerful, which is itself a reason to verify carefully.

Action required before publication: Obtain the source document and verify before any reference to this comparison appears on the public site.

V8: 15–25% OTA commission rate

NotebookLM said: “Standard OTA commission rates typically range from 15% to 25% globally,” while explicitly stating that this figure was not in the source corpus.

Direct check: The figure is not from the project source set. It is a general industry assertion.

Verdict: Requires external citation. The rate range itself is plausible and consistent with publicly available data from Booking Holdings and Expedia Group annual filings.

Action required before publication: Replace the NotebookLM-derived figure with a directly cited source. Booking Holdings’ and Expedia Group’s most recent annual filings are publicly available and disclose their commission ranges and effective take rates.

What this means for the project

1. Three of the four foundation figures are now verified directly. The 80% unregistered figure, the 24,500/4,000 bed breakdown, and the Levi tourism revenue series 2014–2024 are all verified to original Finnish-language documents with corrected page references.

2. One figure remains pending. The 70/30 FIT/package split. The Lapland regional strategy document needs to be obtained.

3. The verification protocol works. Across two separate research tasks (the original strategic critique and the leakage estimate), direct verification has caught material errors in NotebookLM’s outputs — including a wrong year, multiple wrong page references, and an unsourced commission rate. The cost of verification is real but small relative to the cost of publishing without it.

4. The Lapin Suhdannekatsaus is a genuinely valuable source. The full Levi tourism revenue series 2014–2024, plus comparable data for five other Lapland tourism centres, gives the project a quantitative spine for the leakage estimate that was missing in earlier drafts.

This log will be expanded as further verification is completed. If you spot an error in the verification itself, please raise it. The verification of the verification is part of the work.