Much of the work in Beyond the Light Barrier happens on computers.
We solve equations, compare mathematical models, run numerical simulations, test software, and sometimes deliberately try to make that software fail.
That raises an important question:
When does a simulation actually become scientific evidence?
The answer depends on what we mean by evidence.
A simulation can provide evidence about what a mathematical model predicts. It can help us discover mistakes, compare competing models, identify interesting parameter ranges, and decide whether a physical experiment might be worth attempting.
But a simulation cannot, by itself, demonstrate that nature actually behaves according to the model being simulated.
A simulation can tell us what a model predicts.
It cannot, by itself, tell us that nature follows the model.
What Is a Simulation?
A simulation is a computational representation of a model.
Usually we begin with assumptions about a physical system and describe those assumptions mathematically.
A computer then evaluates those equations under specified conditions.
For example, suppose we want to predict the motion of a charged particle in an electric field.
We can describe the particle's motion using established equations of physics, give the computer values for quantities such as the particle's mass, charge, electric field strength, and starting conditions, and calculate how the model says the particle should move.
The resulting trajectory is a prediction of the mathematical model.
It is not a measurement of a physical particle.
The Model Comes First
Before we can understand what a simulation tells us, we need another term: model.
A scientific model is a simplified description of some aspect of reality.
Models can include equations, assumptions, approximations, boundary conditions, and definitions of the quantities being studied.
Even extremely successful physical theories are represented by models when we apply them to particular situations.
A simulation therefore does not simply ask a computer:
“What will nature do?”
It asks something much more specific:
“Given these equations, assumptions, parameters, and starting conditions, what does this model predict?”
That difference is fundamental.
A Computer Can Give a Precise Answer to the Wrong Question
Computers are extremely good at performing calculations.
But they do not automatically know whether the assumptions behind those calculations are physically correct.
Imagine that we create a mathematical model containing an incorrect assumption.
We could write excellent software.
The program could run perfectly.
Every calculation could be performed with extraordinary numerical precision.
We could produce beautiful graphs.
And the result could still fail to describe nature because the underlying model was wrong.
Correctly calculating the consequences of an assumption does not establish that the assumption is physically true.
So How Do We Know the Simulation Is Calculating Correctly?
This brings us to verification.
In scientific computing, verification asks whether the equations and algorithms are being implemented and solved correctly.
In simpler language:
Did we calculate what we intended to calculate?
There are many ways to investigate that question.
We can compare the program with problems whose answers are already known.
We can calculate the same result using different numerical methods.
We can test limiting cases where the equations should reduce to something simpler.
We can deliberately supply unusual inputs and see whether the program handles them correctly.
We can also examine whether increasing the numerical resolution causes the result to approach a stable answer.
What Is Convergence?
That last idea is called convergence.
Many scientific calculations cannot solve a continuous physical problem exactly. Instead, the computer divides the problem into many smaller numerical steps.
Suppose we calculate a trajectory using 10 steps.
Then we calculate it using 100 steps.
Then 1,000.
Then 10,000.
If the numerical method is behaving properly, we may see the calculated answer approach a stable value as the steps become smaller.
That behavior gives us evidence that the numerical approximation is approaching the solution of the equations we intended to solve.
But notice what convergence does not establish.
It does not establish that those equations describe a new physical phenomenon.
It strengthens confidence in the calculation.
Verification and Validation Are Different Questions
Another important term is validation.
Verification and validation sound similar, but they ask different questions.
Verification:
Did we solve the equations correctly?
Validation:
Does the model adequately describe the physical system we are trying to
understand?
Validation requires comparison with physical observations, measurements, or experiments within the conditions where the model is being evaluated.
This distinction becomes especially important when investigating speculative physics.
A mathematical model might be internally consistent.
A computer implementation might correctly solve its equations.
Multiple numerical methods might agree.
And none of those facts alone establishes that the proposed phenomenon exists in nature.
What About Repeating the Calculation?
Repetition is also important, but not every kind of repetition tells us the same thing.
Suppose we run the same program twice and obtain the same answer.
That can tell us something useful about the program's repeatability.
But both runs could contain exactly the same mistake.
Now suppose we implement the same equations using two different numerical methods.
Agreement between them gives us stronger evidence that a particular numerical method is not solely responsible for the result.
But both implementations may still share the same assumptions or even the same misunderstanding of the equations.
That is why we also care about independent reproduction.
What Does Independent Reproduction Mean?
Independent reproduction means that another researcher or team attempts to reproduce a result without merely rerunning the original calculation exactly as it was provided.
Ideally, they independently interpret the equations, implement the method, perform the calculation, and compare their result with the original.
If they obtain the same result, confidence in the computational finding increases.
If they obtain a different result, that is scientifically useful too.
The disagreement tells us that something needs to be investigated.
Reproduction is not valuable because everyone is supposed to agree.
It is valuable because disagreement can expose something we missed.
Where Does BTLB-SIM Fit Into This?
BTLB-SIM is the computational framework being developed for Beyond the Light Barrier.
Its purpose is not to make speculative physics look convincing.
Its purpose is to help us ask computational questions in a way that preserves the information needed to understand, verify, challenge, and potentially reproduce the results.
That includes things such as:
- the equations being solved,
- the assumptions being made,
- units and reference frames,
- parameters and starting conditions,
- the numerical method,
- the validity limits of the model,
- uncertainty information,
- software and model versions,
- provenance showing how a result was produced, and
- verification records.
Provenance is simply the record of where a scientific result came from—what model, software, inputs, assumptions, and processes produced it.
Without that information, a number produced by a computer can become very difficult to evaluate scientifically.
Why Are We Trying to Break Our Own Simulator?
This is also why some BTLB-SIM development has involved deliberately trying to make the system fail.
We want to know what happens if a calculation is interrupted.
What happens if an artifact is corrupted?
What happens if part of a scientific record is incomplete?
What happens when unusual or hostile inputs are supplied?
What happens when different processes try to change something at the same time?
These tests do not tell us whether wormholes or warp drives exist.
They test whether the computational infrastructure can preserve a trustworthy record when things do not go according to plan.
Reliable scientific software does not make a speculative physical idea
true.
It helps us determine more reliably what the idea actually predicts.
Then What Is a Computational Experiment?
We sometimes use the term computational experiment.
That does not mean a physical experiment happened inside the computer.
A computational experiment systematically varies parameters, conditions, or models and observes the resulting numerical behavior.
In that sense, we can experiment with the mathematical model.
But the object being tested is still the model and its computational consequences.
No physical wormhole appears because a wormhole simulation runs.
No spacecraft travels faster than light because a warp geometry is calculated.
No particle accelerator has been operated because we simulated a particle trajectory.
Computational experiments and physical experiments answer different questions.
What Counts as Physical Experimental Evidence?
Eventually, if a model makes a measurable prediction and the experiment is scientifically and practically justified, we can test that prediction in the physical world.
A physical experiment uses actual apparatus, instruments, controls, calibration, and measurements to observe what happens in nature.
Even then, one surprising measurement is not automatically proof of an extraordinary new phenomenon.
We have to consider measurement uncertainty, instrument behavior, conventional explanations, environmental effects, experimental controls, and whether the result can be reproduced.
But this is the stage where the question fundamentally changes.
We are no longer asking only:
“What does our model predict?”
We can begin asking:
“What did nature actually do?”
A Simulation Can Still Be Important Evidence
None of this means simulation results are unimportant.
Quite the opposite.
Computational results can provide strong evidence about mathematical and numerical questions.
They can show that a proposed model behaves differently than expected.
They can expose instabilities.
They can reveal enormous energy requirements.
They can identify parameter ranges where an effect becomes too small to measure.
They can eliminate proposed experiments before time and money are spent building them.
And sometimes the most valuable result of a simulation is:
“This probably isn't going to work.”
That is still scientific progress because it constrains where we should look next.
The Evidence Should Match the Claim
This leads to the principle we try to preserve throughout Beyond the Light Barrier:
The evidence should match the claim.
If we have performed an analytic calculation, we should call it an analytic calculation.
If we have performed a numerical simulation, we should call it a numerical simulation.
If someone independently reproduces it, we should record that separately.
If we eventually perform a physical experiment, that should be identified as physical experimental evidence.
Moving from one category to another requires additional evidence.
Excitement about the result does not move it up the ladder.
So When Is a Simulation Evidence?
The answer is:
Once the simulation has been sufficiently verified for the claim being made—but only for the claims it actually supports.
A properly constructed simulation can provide evidence about the consequences of a mathematical model.
Verification can increase our confidence that the calculation was performed correctly.
Independent reproduction can increase our confidence that the result is not merely an artifact of one implementation.
Comparison with physical measurements can begin testing whether the model describes nature.
Each stage answers a different question.
Keeping those questions separate may make extraordinary research sound less dramatic.
But it makes the conclusions much more meaningful.
The computer can tell us what our model says.
Nature gets the final answer.
Research Status
Current status: This article explains how numerical and simulation results are evaluated within the Beyond the Light Barrier Research Method. It does not report a new physical discovery.
- Evidence Category 7 — Numerical/simulation results: Simulations can provide computational evidence about the behavior of a specified model once they have been sufficiently verified for the claim being made. Such results remain conditional on the model assumptions, equations, parameters, boundary conditions, and numerical implementation.
- Evidence Category 8 — Independently reproduced results: Independent reproduction can strengthen confidence that a computational result is not specific to one implementation, codebase, numerical method, or investigator. This article does not itself report a new independently reproduced result.
- Evidence Category 9 — Experimental results: Comparison with physical measurements or experiments is required to test whether a model adequately describes nature within the conditions being evaluated. This article does not report a new physical experimental result.
A verified or independently reproduced simulation should therefore be interpreted as evidence about the specified computational model at the level justified by its verification and reproduction status—not as evidence that a speculative physical phenomenon necessarily exists in nature.
Sources & Further Reading
ASME. Verification, Validation, and Uncertainty Quantification Terminology in Computational Modeling and Simulation (VVUQ 1-2022). American Society of Mechanical Engineers.
Oberkampf, William L., and Christopher J. Roy. Verification and Validation in Scientific Computing. Cambridge University Press, 2010. DOI: 10.1017/CBO9780511760396.
Roache, Patrick J. “Quantification of Uncertainty in Computational Fluid Dynamics.” Annual Review of Fluid Mechanics, Vol. 29, 1997, pp. 123–160. DOI: 10.1146/annurev.fluid.29.1.123.
Beyond the Light Barrier. Beyond the Light Barrier Research Method, Version 1.1. Project research-method framework for evidence classification, reproducibility, simulation, and experimental interpretation.
No comments:
Post a Comment