Source(s) of Truth

Previous Chapter

Short Answer

“Preoccupied with a single leaf… you won’t see the tree. Preoccupied with a single tree… you’ll miss the entire forest. Don’t be preoccupied with a single spot. See everything in its entirety… effortlessly… that is what it means… to truly ‘see.’”

  • Soho Takuan to Musashi Miyamoto, Vagabond (2003, Translation)

Even if you are the innovator and/or pioneer of the subject/field you are teaching, you won’t know everything. Expect to use external sources to reinforce and supplement teaching.

If you find someone who knows and can do everything and anything, please tell me so I can add them to my network and get an excellent professional reference for my future career path.

Last, but not least, your education system very likely doesn’t need a shiny new curriculum every time they appear or 12+ different online platforms and tools for learning.

Long Answer

Despite this being a book on education systems (and education in general) and any experience in the field of education, I’ll admit I’ve read more management books than education books. Funny how that works.

I’d also reasonably assume education will move away from hard copy (i.e. “physical”) textbooks and other sources of information the more we digitalize and store information online and through databases instead, mainly for cost and practical reasons. This has already happened at large by the time of this writing, with expectations it’ll be more prevalant in the future despite any resistance towards it.

There are two sources of truth to work with: the knowledge you’ve already acquired vs the knowledge you need to acquire.

For both ways, I like these two quotations:

“Absence of evidence is not evidence of absence”

  • Mr. Dugald Bell (Sheppard, 1895, p. 132)

“Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.”

  • Former Secretary of Defense Donald Rumsfeld (U.S. Department of Defense, 2002)

There are three explicit categories stated here: known knowns, known unknowns, and unknown unknowns. A fourth category was observed and added by Steve Rayner (2012), called “uncomfortable knowledge,” are unknown knowns. To sum each category:

Known Known: Knowledge and skills gained through effort and time.

Known Unknown: Familiarity with areas you lack detailed understanding on (breadth vs depth).

Unknown Known: Biases, judgments, and unreviewed or unchecked beliefs.

Unknown Unknown: Information neither known or anticipated.

Exploring Outside Your Field

Though Rumsfeld mentions unknown unknowns as difficult, I’d also categorize unknown knowns as equally challenging. Beliefs, suppositions, and practices we do not know of may form the values we know of, follow, and practice.

For your curriculum and meeting standards, you’ll be dealing with mainly known knowns and known unknowns. You’re able to teach because you know what you’re teaching. For parts you’re uncertain about on the subjects you’re covering (known unknowns), you’re able to find and research resources elsewhere, make sense of them, and incorporate them into your teaching.

In the process of researching, you’ll eventually come upon information that you may or may not use in teaching, but didn’t know about before, existed, or initially thought relevant to your subject(s) (unknown unknowns). Funny how intelligence work is helpful in teaching, too.

Say, for example, you’re teaching introduction to engineering. Considering looking into examples from unrelated fields like management, finance, communication, and more. Tie these fields into your current material, showcase new perspectives, and show real world applications reinforcing why the information matters and how what you’re teaching is used elsewhere.

Here are some books I’ve used to help design curriculum and lesson plans. Despite my credentials and focus on science and engineering, I found insights in various types of literature.

  • Brown, P. C., Roediger, H. L., III, & McDaniel, M. A. (2014). Make it stick: The science of successful learning. Harvard University Press. (Amazon Link)
  • Oakley, B., Rogowsky, B., & Sejnowski, T. (2021). Uncommon sense teaching: Practical insights in brain science to help students learn. TarcherPerigee. (Amazon Link)
  • Duarte, N. (2010). Resonate: Present visual stories that transform audiences. John Wiley & Sons. (Amazon Link)
  • Hall, E. (2024). Just Enough Research. Mule Books. (Amazon Link)
  • Knapp, J., Zeratsky, J., & Kowitz, B. (2016). Sprint: How to solve big problems and test new ideas in just five days. Simon & Schuster. (Amazon Link)
  • Krug, S. (2014). Don’t make me think. Sensible. (Amazon Link)
  • Greever, T. (2015). Articulating design decisions: Communicate with stakeholders, keep your sanity, and deliver the best user experience. O’Reilly Media. (Amazon Link)
  • VandeHei, J., Allen, M., & Schwartz, R. (2022). Smart brevity: The power of saying more with less. Workman Publishing. (Amazon Link)
  • Brewer, C. A. (2016). Designing better maps: A guide for GIS users. Esri Press. (Amazon Link)

Textbooks in General

After reading and reviewing textbooks to use for my own lesson plans, I came to this conclusion: they’re accounting for multiple sets of standards simultaneously.

In the “Legality” chapter, I talked about how the responsibility of education generally falls upon each individual state, like Colorado, New York, etc. That affects textbooks as well.

If a textbook is published in the United States, and is catered towards American curricula, it wants to cover as many standards from each state as possible in a single book. This may be done because it’s cheaper to have 1 textbook rather than 50 textbooks, but also because it keeps all information inside of one book. There may be cases where states are large enough (e.g. California) where designing a specific book for that’s states standards may make sense from a cost and learning perspective.

A textbook designed at a national level and not at the state level is one reason why supplementary material may be required to meet the standards and additional requirements imposed by states, districts, or even counties and cities where instruction takes place. It also helps explain why teachers and instructors may use and create specially designed materials for teaching, rather than textbooks, for their classroom.

Though the textbook may not provide all necessary information (or even too much information), they’re still a valid source of truth to satisfy the standards and objectives teachers and instructors need to cover. Because they do provide information to satisfy multiple standards, it’s not always necessary for a teacher to fully cover every detail inside of one textbook for a given class; especially if some parts don’t align with your standards.

  • For 1:1 instruction, specialized groups, and private tutoring, the entire textbook may be covered instead, but that’s more doable for that specific environment compared to typical classrooms.

As an aside, also be careful when textbooks come with new “editions.” These can range from changing a couple things in errata to complete rewrites. Sometimes this means staying up to date means repurchasing textbooks (which the companies writing and publishing likely want because what business doesn’t want money, right?). This could also mean an old textbook from 10+ years ago still meets standards with a bit of scaffolding and supplements, many of them potentially free, added in here and there.

  • To give a “fun” example of when you should upgrade editions: a tabletop RPG with Pathfinder 1e vs Pathfinder 2e.
    • Multiple, significant changes exist between the two editions, so you should get a copy of the new edition to stay updated and accommodate players who need/want to use the new rules and systems.

The “PDF” of a Book (and its Citations)

Re: Alternate version of class materials.

If a technological update renders PDFs obsolete, pretend its the new thing instead here. I still think Ctrl+F (Control+F) on a keyboard may solve many problems, but that’s me.

You may have a specific textbook (or assigned materials) for a particular class. It is possible there are other versions of all those materials available for use.

If you’re in a situation where a student acquires an alternate version of said material(s), instructors are usually within their station to ask for proof of its existence. This could be showing it physically, submitting a PDF copy through an email, or similar verification for explicit proof.

You want to mitigate students falsifying records and having citation problems down the line. The alternate version may have different page numbers, different citations, alterations in text and visuals, and more. Even if students use an alternate version, they must still cite that version as a source, prove its existence is traceable, and show the information wasn’t acquired through illegal means.

As for policies which may satisfy accommodations:

  1. Ask for the PDF (or similar material) used as a source.
  2. Request specific page numbers (or line numbers) as part of a source.

If material is improperly cited, it could fall under academic dishonesty, cheating, and more which merit hefty consequences.

Regarding Wikipedia

Here are some benefits to Wikipedia:

  • An effective starting point to learning about multiple topics
  • Some high-profile or sensitive articles on Wikipedia are under strict protections
  • Ways to automatically revert vandalism
  • Articles can track revision histories
  • Strict citation requirements for certain information, like quotes

There are also reasons why I criticize using it as a source for essays and papers:

  • Open contribution and editing of articles is still risky
  • Some topics may have data validity issues
  • Subject to heavy bias from contributors
  • Wikipedia may be less accurate than the academia/research it claims to cite from

Additionally, a Wikipedia article does not hold the same weight as another primary source, even if they have the exact same information, because you normally shouldn’t cite encyclopedias anyway.

Despite the primary source issue, I would genuinely prefer a citation from Wikipedia over an AI-generated response used as a citation. You could link back to search engine results (like I’ve done in the Roadblocks chapter). That’s still “fine” to do if pointing to an example of where further research can be done. You’re not supposed to pull out the answers a search engine gives you though; a search engine finds citations you then look into and use as sources.

  • While you’re not supposed to cite encyclopedias over original research and other primary sources, at least Wikipedia has revision histories, discussion methods, and is checked quite often by humans (as of originally writing this on December 2025).

Despite Wikipedia’s benefits and drawbacks, a student may inevitably use it, if not at least look at it. To counter this, consider implementing the following for work where students do research:

To emphasize again: I find Wikipedia better suited as a reference after you have base knowledge of topics, or exploring topics related to what you already know, rather than a resource for learning new material. The website’s goal is not on pedagogy, or even andragogy; it’s an encyclopedia (which are not ideal for learning new topics).

  • From personal experience, with some bias, on my end: Wikipedia for mathematics. You’re better off reading a textbook, or another resource outlining a defined path, and doing math rather than reading about math on Wikipedia for learning purposes.

The Dark Side of the Moon

What should work in theory doesn’t always work as well in practice.

Academia operates under controlled settings and takes into account many assumptions, such as good-faith effort on students (if they’re part of the group tested for research). It’s precisely because it’s controlled conditions, which can be replicated and reproduced for veracity, it’s considered valid as research.

That doesn’t mean the research itself is perfect. Even research groups who made one study can, and should, refute their findings based on later evidence.

As for how old research can be while remaining relevant, it’s a somewhat complicated answer. I’ve seen methods around for several decades that, quite frankly, work well but have relatively dated sources. I’ve also seen new research crop up as a “fad” before fading away from existence as a mistake to hopefully not repeat again. It’s also dependent on the field; a math book from 100 years ago could still be relevant today, pending some minor changes to its explanations, whereas some science books could be made irrelevant tomorrow if a major discovery comes out of nowhere.

  • This may be formally referred to as part of the Lindy Effect (Taleb, 2012).

Because the conditions are controlled, however, this may lead to “Luxury Beliefs” which is ideas suitable for people with resources vs people without enough resources, or upper class vs lower class (Henderson, 2022). Research is always at risk of being divorced from reality and this extends beyond education across multiple fields. The same policies and ideas may have positive effects in one environment, but drastically negative effects in another environment, which runs counter against providing consistent equity for education in the first place!

With research, there may also be issues with p-hacking, replication failures, variable control, correlation, and bias inserted into the research (Ulrich & Miller, 2020). There may also be business incentives and programs affecting the research to showcase “the next best thing” and create evidence for it to generate a profit on. This can also mean “hunting” for studies supporting existing opinions and bias to push forward a policy or method.

Additionally, some people conducting the research on education and classrooms may have never taught in a classroom before, which may remove vital context to inform their findings and draw conclusions. This lack of content may or may not be their fault either; acquiring data on students in education has many barriers and ethical concerns attached which may dilute the research. What that means is they may acquire samples either too small for statistical significance or unrepresentative of populations to reproduce and verify research methods.

All the theory in the world cannot fully correct problems at a real-life, systemic level. At its best, the theory is implemented and can solve a set of problems in a class, but not the entire classroom system as a whole at varying success. At its worst, the theory falls apart completely, or is adjusted to suit the interests of its implementers, and may make the classroom environment much worse.

Not all educational research is bad. Some research is good and research on how to make a more effective and efficient classroom is something each party should have an interest in, including teachers. Nevertheless, be mindful of how research is conducted and how well it translates to real-life scenarios.

The Panel Council

Perhaps you’re a teacher, instructor, or something similar and want to explore content you have some familiarity with. Perhaps you have no familiarity with the content itself, but you are familiar with the system, its rules/laws, and the process to integrate content from outside into the system.

This is where a panel of experts, or subject matter experts (SMEs) comes in. To stay concise, they are individuals with understanding of a specific field. You may need one of these people for examples such as:

  • A virologist making corrections to your explanations on infectious diseases.
  • An aviation expert checking the algorithms and formulae for aerodynamics and physics.

Straightforward ways to identify subject matter experts are through an advanced degree (Master’s or PhD), years of experience (Senior vs Junior), and/or licensure (a non-licensed architect vs a licensed architect). You could also include inventors of a particular tool or software, such as the inventor of hammers being a subject matter expert on hammers… because they made it.

If these experts are not available, usually the next best source becomes people who may not have any of these qualities, but developed a vested interest into the content you’re working on. Examples for this are harder to describe, but perhaps someone with a thousand hours into Pathfinder Second Edition is more knowledgeable about the TTRPG vs someone who just started Pathfinder.

These experts you should happily accept into helping design an education system, or even a curriculum, because they’ll verify facts with certainty vs a non-expert verifying facts with some certainty instead. Even if you don’t directly enlist their help, they may have resources you can cite from to better design your own systems or curricula.

Bibliography

  1. Brewer, C. A. (2016). Designing better maps: A guide for GIS users. Esri Press. https://www.amazon.com/Designing-Better-Maps-Guide-Users/dp/1589484401
    • Further Reading Source
  2. Brown, P. C., Roediger, H. L., III, & McDaniel, M. A. (2014). Make it stick: The Science of Successful learning. The Belknap Press of Harvard University Press.
    • Further Reading Source
  3. Duarte, N. (2010). Resonate: Present visual stories that transform audiences. John Wiley & Sons, Inc.
    • Further Reading Source
  4. Greever, T. (2015). Articulating design decisions: Communicate with stakeholders, keep your sanity, and deliver the best user experience. O’Reilly Media, Inc.
    • Further Reading Source
  5. Hall, E. (2024). Just Enough Research. Mule Books. https://www.amazon.com/Just-Enough-Research-Erika-Hall-ebook/dp/B0DJWS2SL1
    • Further Reading Source

  6. Harvard Library. (n.d.). Evaluating Sources. https://usingsources.fas.harvard.edu/evaluating-sources-0

  7. Henderson, R. (2022, June 12). Luxury Beliefs are Status Symbols. www.robkhenderson.com. https://www.robkhenderson.com/p/status-symbols-and-the-struggle-for

  8. Inoue, T. (2003). Vagabond (Vol. 4). VIZ Media LLC. (Original work published 1999). ISBN: 978-1-56931-854-6

  9. Kansas State University Libraries. (n.d.). Evaluate Sources. https://guides.lib.k-state.edu/c.php?g=181829&p=1195993

  10. Knaflic, C. N. (2025). Storytelling with data. https://www.storytellingwithdata.com/

  11. Knapp, J., Zeratsky, J., & Kowitz, B. (2016). Sprint: How to solve big problems and test new ideas in just five days. Simon & Schuster. https://www.amazon.com/Sprint-Solve-Problems-Test-Ideas/dp/150112174X
    • Further Reading Source
  12. Krug, S. (2014). Don’t make me think. Sensible. https://sensible.com/dont-make-me-think/
    • Further Reading Source

  13. McCandless, D., et al. (n.d.). Information is beautiful. https://informationisbeautiful.net/

  14. Microsoft. (n.d.). Power BI. https://powerbi.microsoft.com/en-us/

  15. Oakley, B., Rogowsky, B., & Sejnowski, T. (2021). Uncommon sense teaching: Practical insights in brain science to help students learn. TarcherPerigee.
    • Further Reading Source

  16. Python Graph Gallery. (n.d.). The Python graph gallery. https://www.python-graph-gallery.com/

  17. Rayner, S. (2012, February 9). Uncomfortable knowledge: The social construction of ignorance in science and environmental policy discourses: Economy and society: Vol 41, no 1. Taylor & Francis Online. https://www.tandfonline.com/doi/abs/10.1080/03085147.2011.637335

  18. Sheppard, T. (1895, December). On the occurrence of Scandinavian boulders in England. The Glacialists’ Magazine: A Quarterly Record of Glacial Geology, 3(3), 129–132. F. H. Butler. https://babel.hathitrust.org/cgi/pt?id=uc1.b4182734&seq=152

  19. Taleb, N. N. (2012). Antifragile: Things That Gain from Disorder. Random House. ISBN: 1-400-06782-0.

  20. Ulrich, R., & Miller, J. (2020). Questionable research practices may have little effect on replicability. ELife, 9. https://doi.org/10.7554/elife.58237

  21. United States Department of Defense. Defense.gov Transcript: DoD News Briefing - Secretary Rumsfeld and Gen. Myers. (2002, February 12). https://web.archive.org/web/20160406235718/http://archive.defense.gov/Transcripts/Transcript.aspx?TranscriptID=2636

  22. VandeHei, J., Allen, M., & Schwartz, R. (2022). Smart brevity: The power of saying more with less. Workman Publishing.
    • Further Reading Source
  23. Watts, M. (2024, October 14). Subject Matter Experts: Key Players in Process Improvement and Business Success. iSixSigma. https://www.isixsigma.com/dictionary/subject-matter-expert-sme/

Next Chapter