Research Journal Entry 17 — What Happens When We Try to Break It?

There comes a point in developing scientific software when passing the tests you already know how to write is no longer enough.

Eventually, you have to start asking a different question:

What happens if we deliberately try to break it?

That is the question behind the next stage of BTLB-SIM development: AR-1.7, the Architecture Stress Audit.

The objective is not to prove that BTLB-SIM works perfectly.

It is almost the opposite.

We are deliberately looking for circumstances in which the architecture fails, produces an incomplete scientific record, loses information, behaves inconsistently, or allows something to happen that the research framework should have prevented.

And when we find those failures, we are preserving them.

From Portability Testing to Stress Testing

The previous stage of the project asked whether important portions of BTLB-SIM could behave consistently across macOS and Windows.

That work uncovered several assumptions that had been hidden while the software was being developed primarily on one operating system.

Those failures were useful.

They demonstrated something that is easy to forget in software development: a system can appear reliable simply because it has not yet encountered the conditions that expose its weaknesses.

Cross-platform verification deliberately changed some of those conditions.

AR-1.7 takes the idea considerably further.

Instead of asking whether the system works when everything behaves normally, we are intentionally creating situations in which things may go wrong.

A Frozen Baseline

Before beginning the stress audit, we needed to know exactly what system we were testing.

That means the audit begins from a defined baseline rather than from software that is continually changing underneath the investigation.

For AR-1.7 — Architecture Stress Audit, that baseline is the corrected BTLB-SIM software state established by merge commit 4543b1f. Its merged-main acceptance verification is recorded by commit 2e3af1d, which documents the closure boundary of the preceding cross-platform portability-correction campaign and identifies AR-1.7 as the next architecture milestone.

These identifiers are part of the audit provenance. They distinguish the software baseline being stressed from later changes that may result from what the audit discovers.

The AR-1.7 audit specification defines 30 case IDs designed to probe different kinds of architectural stress.

They include areas such as:

• concurrency and race conditions,
• interrupted execution and recovery,
• artifact and provenance corruption,
• resource exhaustion,
• hostile or malformed inputs,
• dependency and module isolation,
• determinism and reproducibility, and
• attacks on the integrity of the scientific record itself.

These are not 30 attempts to test wormhole physics.

They are attempts to determine whether the computational framework we may eventually use for difficult scientific questions can preserve its own scientific rules when conditions become unfavorable.

Why We Are Not Fixing Things Yet

One of the stranger decisions in this stage of the project is also one of the most important:

When we discover a problem, we are not immediately fixing it.

That may sound backwards.

Normally, when a programmer discovers a bug, the natural response is to fix the bug and continue.

But AR-1.7 is currently a discovery exercise.

If we change the production software every time we discover something, then the system being tested gradually changes while the audit is still underway.

A later stress case might then be testing a different architecture from the one encountered by an earlier case.

Worse, the original failure could become difficult to reconstruct after the correction has been made.

So discovery and remediation are deliberately separated.

Discover → Preserve → Reproduce → Classify → Correct later

The objective is to understand the failure before changing the system that produced it.

The Audit Is Already Finding Things

AR-1.7 is still in progress, but the stress audit has already reproduced six findings within the project-controlled audit environment.

Here, reproduced means that the observed software or architecture behavior could be produced again during the AR-1.7 investigation. It does not mean that the findings have been independently reproduced by an external researcher or independently developed implementation, and it does not qualify as Beyond the Light Barrier Evidence Category 8.

They involve:

• descriptor mutation,
• incomplete completion records,
• checkpoint history,
• partial ledger registration,
• import detection, and
• test discovery.

These findings do not all necessarily represent the same kind or severity of problem.

That is one reason classification comes after discovery rather than being assumed from the moment something unexpected occurs.

Additional variants and native-Windows execution remain pending as part of the ongoing investigation.

No production corrections have yet been proposed or implemented as part of this discovery stage.

Even the Test Can Fail

There is another important rule in the audit:

Failures in the test harness are failures worth preserving too.

A stress test is itself software.

That means the mechanism designed to expose a weakness can contain weaknesses of its own.

Simply discarding those cases because “the test failed” would remove potentially useful information from the record.

AR-1.7 therefore retains and hashes failures, including harness failures, rather than silently removing inconvenient runs from the investigation.

During the current discovery work, the baseline files have remained unchanged, and the preserved VAPE and WC archives have continued to pass their verification checks.

That gives us an important reference point while the architecture is being stressed.

Trying to Damage the Scientific Record

Some of the most interesting AR-1.7 cases are not primarily concerned with whether a calculation crashes.

They are concerned with whether the scientific record remains trustworthy.

Suppose a calculation completes, but its provenance record is incomplete.

Suppose a checkpoint exists, but the history needed to understand how it was produced is lost.

Suppose only part of a result is registered before an interruption occurs.

A program might still appear to have “worked.”

Scientifically, however, the result may no longer be trustworthy enough to use.

This is one of the reasons BTLB-SIM treats provenance, completion records, artifact identity, and verification as part of the scientific architecture rather than as administrative details added after a calculation finishes.

Failure Is Not the Opposite of Progress

Earlier in this project, we adopted the principle that negative scientific results should be preserved.

The same principle applies to the software supporting the research.

If a stress case exposes a weakness, the correct research response is not to hide the failure so that the software appears more mature.

The failure has identified something we did not adequately understand before.

That makes it information.

A system that survives every test we know how to perform may be strong.

Or we may simply not have learned how to test its weak points yet.

AR-1.7 is an attempt to become better at finding those weak points.

What This Does Not Tell Us

Architecture stress testing can strengthen confidence in the computational framework used by the project.

It can reveal hidden assumptions.

It can expose incomplete provenance, recovery problems, nondeterministic behavior, and other weaknesses that could compromise future computational research.

But none of those results demonstrates new physics.

Finding a software weakness is not finding a new physical phenomenon.

AR-1.7 does not provide evidence for wormholes, anomalous gravity, warp geometries, faster-than-light communication, or faster-than-light propulsion.

It also does not constitute independent scientific reproduction or a physical experiment.

This is software and architecture verification intended to make later computational research more trustworthy.

Writing the Journal Before We Know the Ending

There is another reason for documenting AR-1.7 now rather than waiting until the audit is finished.

We do not yet know what the final result will be.

More problems may be discovered.

Some current findings may turn out to have simple explanations.

Others may require architectural changes.

Corrections may expose additional problems when the verification gates are run again.

Recording the state of the investigation now prevents us from accidentally rewriting the history later with knowledge we did not yet possess.

At this point in the project, the record is straightforward:

AR-1.7 is in progress.

Thirty stress cases have been defined.

Six findings have been reproduced so far.

Discovery is continuing.

Production corrections have not yet begun.

What Comes Next

Once the discovery phase is sufficiently complete, the findings can be classified and evaluated.

Corrections can then be designed without erasing the behavior that motivated them.

After corrections are implemented, the appropriate regression and cross-platform verification gates will need to be run again.

And if something fails again, that failure belongs in the record too.

The objective is not to create software that has no recorded failures.

The objective is to create a research framework in which failures are difficult to hide and useful to learn from.

Sources & Further Reading

• Beyond the Light Barrier Research Method v1.0 — project research methodology, evidence categories, verification requirements, and rules for preserving negative and null results.
• BTLB Software Development Standards v1.0 — project standards for scientific software development, provenance, reproducibility, portability, verification, and preservation of historical results.
• BTLB-SIM Cross-Platform Verification Profile — project development record documenting the cross-platform portability verification campaign, merged-main acceptance, corrected BTLB-SIM baseline, and transition to AR-1.7 — Architecture Stress Audit.

Research Status

• BTLB-SIM: Active development.
• AR-1.7 — Architecture Stress Audit: Discovery phase in progress against the frozen BTLB-SIM baseline identified in this entry.
• Current audit state: 30 stress cases defined; six findings reproduced within the project-controlled audit environment; classification and remediation are not yet complete.
• Evidence Category 7 — Numerical/simulation results: No new physics simulation result is established by this architecture audit.
• Evidence Category 8 — Independently reproduced results: No new Category 8 result. Reproduction of software findings within AR-1.7 is internal audit reproduction, not independent scientific reproduction.
• Evidence Category 9 — Experimental results: No new Category 9 result. AR-1.7 is not a physical experiment.
• Physical interpretation: Nothing reported in this entry is physical evidence for wormholes, anomalous gravity, warp geometries, faster-than-light communication, or faster-than-light propulsion.

Before we trust BTLB-SIM with increasingly difficult questions,
we should first become very good at trying to break it.

Research Journal Entry 16 — When “Works on My Computer” Isn’t Good Enough

Scientific computing has a familiar software-engineering problem:

It works on my computer.

For ordinary software development, that phrase is usually a warning that something about the program depends on the particular machine or environment in which it was developed.

For scientific software, the problem can be more serious.

If a computational result depends unexpectedly on the operating system, filesystem behavior, text representation, or some other platform-specific assumption, then reproducing the calculation on another computer may not mean what we think it means.

BTLB-SIM recently encountered exactly this kind of problem.

What initially looked like a software-portability issue became a useful test of several principles we have been trying to build into the project: preservation of failures, provenance, versioning, verification, and careful separation between software reproducibility and scientific evidence.

Why Another Operating System Matters

Much of the early BTLB-SIM development has taken place on macOS.

That is perfectly adequate for developing software, but it creates an obvious question:

What happens when the same computational experiment is moved to a different operating system?

Windows was therefore introduced as another verification environment.

The objective was not to prove any new physics.

It was to determine whether important portions of the computational framework behaved consistently when moved away from the environment in which they were originally developed.

That distinction matters.

Running the same implementation on another operating system can strengthen confidence in the portability and reproducibility of the software.

It does not constitute an independent derivation of the physics, an independently developed scientific implementation, or a physical experiment.

The First Portability Problem

One of the first issues involved filesystem behavior.

Some durability and synchronization behavior had been implemented using assumptions appropriate to POSIX-style systems such as macOS.

Those assumptions could not simply be carried over to Windows.

This is a good example of why portability testing matters.

The scientific model may be unchanged.

The equations may be unchanged.

The experiment definition may be unchanged.

Yet implementation details beneath them can still behave differently on another operating system.

The solution was not to pretend the difference did not exist.

The architecture had to distinguish the scientific requirement from the platform-specific mechanism used to satisfy it.

Then We Found a More Subtle Problem

Another issue involved something that normally seems almost trivial: line endings.

Text-file line-ending conventions can differ across operating systems and tool configurations.

A line ending commonly represented by a single line-feed byte on one system may be represented by a carriage-return and line-feed pair on another.

To a person reading the file, the text may appear identical.

To a cryptographic hash operating on the raw bytes, however, those files are different.

That matters when hashes are being used to identify frozen scientific artifacts.

If the bytes change, the identity check changes—even when the visible text appears unchanged.

This transformed what might otherwise have been dismissed as a Windows text formatting problem into a provenance and reproducibility problem.

If we say that two computational runs used the same frozen specification, we need a sufficiently precise definition of what “the same” means.

The First Windows Run Failed

The initial Windows-targeted verification did not produce a clean result.

1 test passed. 8 tests failed.

That result is important enough to preserve.

The failures were not evidence that the underlying physics was wrong.

They were also not evidence that Windows was unsuitable for scientific computing.

They exposed a problem in how the historical implementation interacted with Windows-specific behavior.

In particular, the tests were invoking a historical V1 implementation that contained the preserved Windows defect.

Once the problem was understood, it would have been easy to fix the old code, rerun the tests, and allow the failed attempt to disappear from the story.

We deliberately did not do that.

The failed run is part of the development record because it tells us something important about the assumptions that existed at that point in the software.

A failure that teaches us something should not disappear merely because we later learn how to fix it.

Correcting the Software Without Rewriting Its History

The correction raised another question:

What should happen to the implementation that contained the defect?

For BTLB-SIM, the answer was not simply to overwrite it and behave as though the earlier implementation had never existed.

The historical V1 behavior was preserved.

A distinct V2 successor corrected the raw-byte specification-verification behavior.

This distinction matters because scientific provenance includes more than the latest version of the software.

Provenance—the traceable record of the models, assumptions, data, parameters, software versions, and methods that produced a result—also requires us to know which implementation existed when an earlier result was produced.

Silently changing historical software can make that reconstruction much more difficult.

Preserving the old implementation while correcting the successor allows both statements to remain true:

This is what the historical implementation did.

This is how the successor implementation corrected it.

Testing the Successor

After separating portable successor verification from direct comparisons against the historical implementation, the recorded test results became:

These results belong to the specific V2 successor implementation and cross-platform verification scope described here. They should not be treated as verification of every BTLB-SIM version, module, operating system, or future implementation. The associated development record should preserve the implementation version or source-code commit, test configuration, execution environment, and verification results needed to identify and reproduce this verification state.

macOS: 16 passes

Windows: 9 passes, 7 intentional skips, 0 failures

The seven Windows skips are important.

They should not be described as successful tests.

They were intentional exclusions involving historical V1-to-V2 execution comparisons that were not appropriate to count as portable Windows passes under the scoped verification policy.

Recording a skip as a skip is part of keeping the verification record honest.

Six preserved scientific regression cases were also compared using exact Python floating-point equality under this particular verification scope.

That does not establish universal numerical identity across every future platform, compiler, solver, or computational environment.

It records what was actually tested under the defined conditions.

What Did We Actually Verify?

This is where evidence classification becomes especially important.

Successful execution on macOS and Windows provides stronger evidence that the tested portions of the implementation are portable across those two environments.

It also helps reveal hidden operating-system assumptions that a single-platform development process might never expose.

But it does not promote the scientific evidence category of the underlying physics.

Cross-platform agreement is internal software verification.

It is not independent scientific reproduction.

It is not a physical experiment.

The distinction is similar to the one discussed in the previous journal entries about running someone else's software.

If another machine executes the same verified computational artifact under defined conditions and obtains the same output, we have learned something valuable about the reproducibility and portability of that implementation.

We have not automatically obtained an independent derivation, an independent implementation, or independent confirmation of the physical model.

Portability Became Part of the Standards

These problems helped clarify that cross-platform behavior could not remain an informal software-development concern.

It needed to become part of the project's documented development standards.

The resulting BTLB Software Development Standards v1.0 formalize requirements involving cross-platform behavior, raw-byte integrity, canonical representation, reproducibility, provenance, structured outcomes, preservation, versioning, verification, and release records.

One particularly useful distinction is that software status should not be collapsed into a single word such as “complete.”

A component may instead need to be distinguished as:

designed → implemented → tested → platform-verified → independently reproduced

Those are different milestones.

Reaching one does not silently imply that the others have also been reached.

There Is Still an Open Problem

The cross-platform work is not finished.

Another issue, tracked as CP-003, concerns operating-system path separators leaking into persistent provenance.

Windows and POSIX-style systems normally represent filesystem paths differently.

If those native representations are written directly into persistent scientific records, logically equivalent provenance may differ merely because the calculation was executed on another operating system.

The intended direction is to use a canonical serialized representation, including consistent forward-slash path notation where appropriate.

But CP-003 remains open in the current record.

We therefore should not claim that BTLB-SIM has completed cross-platform portability verification.

Some important portability problems have been identified and corrected. Others remain under investigation.

Why This Matters to the Physics

None of these issues changes Einstein's equations.

A Windows line ending does not alter general relativity.

A filesystem path separator does not create or destroy a gravitational effect.

But software is increasingly part of the chain through which we calculate, test, preserve, and communicate scientific results.

If that chain cannot tell us exactly what artifact was executed, what version produced a result, what assumptions accompanied it, or whether moving to another platform silently changed something important, then the scientific record becomes weaker.

That is why apparently mundane engineering details can become research-integrity issues.

Where the Project Stands

The current cross-platform work has demonstrated that important portions of BTLB-SIM can be verified across macOS and Windows under the tested conditions.

It has also exposed assumptions that prevented other portions from being portable without correction.

Historical failures and implementations have been preserved rather than silently rewritten.

Successor behavior has been tested separately from historical behavior.

Additional portability work remains open, including CP-003.

No new physical phenomenon is reported by this work.

No wormhole, anomalous gravitational effect, warp geometry, faster-than-light communication, or faster-than-light propulsion has been experimentally demonstrated.

This is a software-verification and research-integrity milestone.

Beyond “Works on My Computer”

The lesson from this stage of BTLB-SIM development is not that scientific software must produce identical behavior everywhere without qualification.

The lesson is that we need to know when it does not.

We need to understand why.

We need to preserve the failures that reveal hidden assumptions.

We need to distinguish historical behavior from corrected behavior.

And we need to state precisely what successful cross-platform verification does—and does not—demonstrate.

“Works on my computer” is the beginning of a verification question, not the end of one.

Sources & Further Reading

Beyond the Light Barrier — Research Method v1.0.
Project methodology governing evidence classification, assumptions, uncertainty, reproducibility, preservation of negative and failed results, corrections, and scientific claims.

Beyond the Light Barrier — BTLB Software Development Standards v1.0.
Project standards governing cross-platform behavior, raw-byte integrity, canonical representation, provenance, versioning, verification, preservation, and release records for BTLB scientific software.

Beyond the Light Barrier — BTLB-SIM Framework Implementation State v0.9-dev.
Development-state record for the BTLB-SIM framework, including implemented capabilities, verification status, and remaining development work.

Research Status

BTLB-SIM: Active development. This entry documents a cross-platform software-verification and research-integrity milestone rather than a new physical result.

Cross-platform verification: Partial. The specific successor implementation and verification scope described in this entry were exercised on macOS and Windows. Additional portability work remains open, including CP-003.

Historical result: The initial Windows-targeted verification produced 1 pass and 8 failures. That result and the historical V1 behavior are preserved as part of the development record.

Successor verification: Under the scope described in this entry, the recorded results were 16 passes on macOS and 9 passes, 7 intentional skips, and 0 failures on Windows. These results apply only to the tested implementation and conditions and should not be generalized to all BTLB-SIM versions or platforms.

Independent reproduction: No new Category 8 result is established by this work. Cross-platform execution of the same implementation is internal software verification, not independent scientific reproduction.

Physical experiment: No new Category 9 result is established by this work.

Evidence status: Nothing reported here constitutes physical evidence for a wormhole, anomalous gravitational effect, warp geometry, faster-than-light communication, or faster-than-light propulsion.