The dry run must be able to fail

The dangerous green

External integrations can appear complete surprisingly easily. According to its test mode, the delivery service accepts every email, the DNS adapter returns a fictitious zone and the simulated payment request reliably ends with success. The interface turns green, the pipeline passes – and yet hardly anything has been proven.

Such a dry run often checks only whether your own code can complete the desired ideal scenario. It does not answer how the system responds to an expired token, a throttled API, a delayed response or an external object that already exists. It becomes particularly problematic when simulation and live operation have separate implementations with different behaviour.

A better basic assumption is this: a simulation is not a convenient substitute for the outside world. It is a controllable implementation of the same domain contract.

This changes the architecture. A global dryRun=true switch should not suppress arbitrary side effects. Instead, every integration boundary needs an explicit interface, defined results and known errors. Only then can simulation, automated testing and the real provider be meaningfully related to one another.

Model the contract first

Application logic should not need to know which SDK is used to communicate with a provider or which URL currently applies. It should use a domain capability. For a deployment process, the contract might look like this:

interface DnsProvisioner
{
    public function createRecord(
        Hostname $hostname,
        IpAddress $target,
        IdempotencyKey $key,
    ): ProvisioningResult;

    public function inspectRecord(Hostname $hostname): RecordState;

    public function removeRecord(
        Hostname $hostname,
        IdempotencyKey $key,
    ): RemovalResult;
}

The types are more important here than the specific syntax. ProvisioningResult should not contain only true or false, but should distinguish between states such as created, already_present, pending and rejected. An error also needs meaning: unauthorised, rate limit, invalid hostname and temporary unavailability require different responses.

TYPO3 supports this separation through its Symfony-based dependency injection system; the current Core documentation recommends constructor injection for service dependencies. Interfaces can therefore point to different implementations depending on the environment. This is not a special testing trick, but regular application architecture. (TYPO3 Dependency Injection)

A domain contract also protects against a common leak: if a service directly returns the response class of a provider SDK, that provider's data model spreads throughout your own code. A later change then affects controllers, jobs, templates and tests. A small result type of your own keeps the translation where it belongs – in the adapter.

Three levels, three different kinds of evidence

“The integration is tested” is too vague. At least three levels answer different questions.

1. Deterministic simulation

The simulation runs without a network, credentials or real infrastructure. It tests the domain process: are records created in the correct state? Is an invitation released only after successful deployment? Can a failed step be retried? Is clean-up planned only for resources that actually belong to the system?

For this purpose, the simulated implementation should be stateful. A DNS record created in the first step must be present the next time it is read. A second identical creation attempt should return already_present – or deliberately produce a conflict if that matches the real contract.

A good simulation can inject errors:

$dns = SimulatedDnsProvisioner::withScenario(
    CreateRecordScenario::rateLimited(retryAfter: 30),
);

This makes it possible to test not only happy paths, but also retries, cancellation and the user interface in a reproducible way. The clock, random number generator and ID generation must likewise sit behind controllable dependencies. Otherwise, a supposedly deterministic test remains dependent on real time or random values.

2. Adapter and contract tests

This level tests the translation between your own contract and the provider protocol. Does the adapter send the correct HTTP method, authentication and payload? Are 429, 401 and malformed JSON classified correctly? Do optional fields actually remain optional?

The Symfony HTTP Client provides MockHttpClient and MockResponse for this purpose. Because the mock implements the same HttpClientInterface as the real client, a test can inspect requests and provide controlled responses, status codes or errors without accessing the network. The documentation explicitly highlights faster, consistent execution and the avoidance of unintended state changes. (Symfony HTTP Client: Testing)

This fits the TYPO3 testing strategy: unit tests isolate individual classes and mock API calls, among other things; functional tests, by contrast, run in a fully configured TYPO3 instance with a database and extension configuration. Both levels are necessary because a correct adapter alone does not prove that service configuration, persistence and permission checks work together. (TYPO3 Extension Testing)

Adapter tests should also check the outgoing request. A hard-coded success object tests only the processing of a response, not the contract with the provider. Stored, sanitised response fixtures are useful as well – provided that they contain no credentials or personal data and that their origin and API version remain documented.

3. Limited live smoke test

No mock proves that current credentials are valid, that DNS and TLS work or that the provider implements its documentation exactly as expected. This requires a small live test against a sandbox or a clearly disposable resource.

However, this test should not be part of every local test run. It requires:

• explicit activation,
• tightly restricted credentials,
• clearly named test resources,
• a known maximum scope,
• reliable clean-up,
• logged external IDs,
• a runtime and cost limit.

The live smoke test proves the connection to the outside world. It replaces neither the broad simulation nor the precise adapter tests. Rare errors such as rate limits or delayed consistency can hardly be forced reliably in a live environment and therefore still belong in controlled scenarios.

A null adapter must not fabricate success

Optional integrations frequently lead to a null implementation. This can be useful: an application remains usable locally even though social publishing, card payments or AI consulting have not been configured.

It becomes dangerous when the null adapter simply reports success. The domain logic then sees no difference between “published externally” and “deliberately did nothing”. Status displays, audit data and subsequent processes tell a false story.

Instead, a null adapter should return an unambiguous result such as disabled. If a feature is enabled in the configuration but the real adapter or its credentials are missing, the application must fail closed. Three states should not be conflated:

disabled   → feature is deliberately inactive
simulated  → process was tested without an external side effect
live       → real adapter was executed in a controlled manner

This information belongs in the result and the log, not merely in an ephemeral environment variable. It must remain possible to determine later whether an email was actually handed over or merely simulated.

Simulation is a runtime environment of its own

A single global dry-run switch quickly becomes unusable in longer processes. Perhaps Lighthouse audits should run for real, while DNS and deployment remain simulated. Or emails should be rendered technically but redirected to a local inbox. The operating mode should therefore be configured for each capability and validated before the process starts.

One possible configuration is:

integrations:
  audit: live
  dns: simulated
  deployment: simulated
  mail: redirected

The application should use this to derive an effective configuration during boot or, at the latest, before the job starts. An enabled live mode without credentials is a configuration error. A production system with accidentally simulated delivery needs at least an unmistakable warning – or, for critical processes, preferably a startup block.

Sensitive values do not belong in simulation logs. Good testability comes from small contracts and controllable responses, not from copying production secrets or complete production data into a local environment.

States must remain correct even after a crash

External side effects and local database transactions generally cannot be combined atomically. A DNS provider may successfully create a record while the subsequent local write operation fails. When the system restarts, it initially does not know whether it should create the record again or merely reconcile the state.

A robust integration therefore needs more than an adapter:

1. Before the call, the intent is stored with a stable idempotency ID.
2. The adapter transmits this ID if the provider supports it.
3. External object IDs and sanitised results are persisted.
4. After an ambiguous outcome, the state is queried before any blind retry.
5. Clean-up operates only on clearly assigned resources.

The simulation must be capable of representing precisely these intermediate states. Otherwise, the most difficult part of the process remains untested.

Observability should reflect the same contract as well. Correlation across process and network boundaries is a core principle of distributed traces; OpenTelemetry describes Context Propagation as the mechanism that allows signals to be correlated across multiple services. In practical terms for an integration, this means that the job ID, correlation ID and – where it is safe to do so – external request ID should be discoverable together. (OpenTelemetry Context Propagation)

Two projects with deliberately controlled seams

In the supplied project information for TYPO3 Demo, every external step has a deterministic simulation: scraping, auditing, generation, DNS, deployment, validation, email and lifecycle. Live DNS, deployment and delivery remain subject to explicit operator approval. An invitation is also tied to the running state of the generated instance. According to the project description, this prevents old or already deleted demo targets from being shared.

In Softwair, external providers are isolated behind interfaces. Social publishing, card payments and AI consulting use null implementations by default. An enabled switch without a verified adapter and credentials must not pretend to have succeeded. The project has been implemented as a foundation but is not operated in production; domains, SMTP, backups and responsible accounts explicitly remain open decisions for the operator.

These are specific properties of the supplied projects, not a claim that every test level described has already been run live for every conceivable provider. The broader lesson is this: the greater the potential external impact, the more clearly simulated, disabled and real execution must be separated.

A practical acceptance checklist

Before an external integration is approved, it should be possible to answer several questions unambiguously:

• Is there a domain contract with no provider classes in its return value?
• Can success, disabled operation, simulation, temporary errors and final rejection be distinguished?
• Can the simulation maintain state and produce targeted errors?
• Do adapter tests check both outgoing requests and incoming responses?
• Does the process run as a functional test in a real TYPO3 instance?
• Is there a limited live smoke test with its own clean-up?
• Are retries idempotent or protected by a prior state query?
• Do the mode, correlation ID and external object ID remain traceable?
• Does an incomplete live configuration fail visibly?
• Can the application prove that it owns a resource before changing or deleting it?

Not every small API requires the same depth. A read-only weather service requires different safeguards from DNS, payments or automated delivery. The structure remains similar, however: the more irreversible, expensive or public a side effect is, the tighter its live contract must be.

The dry run is an alternative to hope

A simulation provides no assurance if it merely returns success everywhere. Its value comes from controlled inconvenience: it must reproduce errors, retain states and deliver the same domain results as the real adapter.

The dry run thus becomes a serious part of the application. It enables broad testing without external impact, while adapter tests verify protocol translation and a few live smoke tests verify the real connection. None of these levels replaces another.

The decisive measure of quality is therefore not: “Can test mode make the pipeline green?” Instead, it is: “Can it show precisely why this pipeline should not turn green in real operation?”