#Physics-inspired speech modelling: a joint direction for Mino and Vũ

Refined 9 October 2026. Status: research hypotheses, not experimental results.

#1. The agreed direction

The research should address speech while using physical characteristics of sound to develop the AI model. Mino contributes speech modelling and evaluation; Vũ contributes the physical formulation, assumptions and analysis. A cardiac application, MeshGraphNet architecture or full wave-equation solver is not required merely to establish the collaboration.

The methodological question is: which acoustic structure should the model learn freely, and which should we build into its representation or computation? Useful candidates include causality, delayed reflections, frequency-dependent energy decay, spatial coherence and the distinction between source-generated reverberation and external noise.

My current recommendation is to investigate a speech enhancement model with an explicit, learnable memory of reverberation. Begin with a temporal mechanism, then test spatial information as an extension. This is a refinement of the earlier speech-preserving dereverberation idea. The previous geometry-uncertainty proposal remains an alternative, rather than the selected direction.

#2. Three physical mechanisms worth investigating

These mechanisms are not new physical discoveries. Their value would come from a specific neural formulation and evidence that it improves speech recovery over strong existing methods.

#3. Recommended hypothesis: give the model a physical memory of echoes

#Intuition

When someone stops speaking, the room may continue to ring. That energy comes from earlier speech. The next sound from the speaker, however, is new information. A useful model should distinguish these two cases instead of treating all temporal correlation as equally helpful or equally removable.

This connects to the existing CorrNet research: inter-frame products reveal relationships between nearby observations, but the relationship may arise from speech periodicity, room reflections or interference. The proposed module gives the network an explicit hypothesis about one of those causes—late reverberation—while retaining a learned speech representation.

#A concrete first model

Use an existing enhancement/dereverberation network with a fixed complex-filter output. Add a small latent state for each frequency band. One illustrative parameterization is:

$$z_{k,f,t}=\alpha_{k,f}z_{k,f,t-1}+\beta_{k,f}q_{f,t-d}, \qquad \widehat r_{f,t}=\sum_k z_{k,f,t}.$$

Here, $q$ is estimated speech power, $z$ stores a decaying contribution from earlier speech, $d$ is a prediction delay, and $\widehat r$ estimates late-reverberation power. Constrain $0\leq\alpha<1$ and $\beta\geq0$. Multiple states allow several decay rates instead of forcing every room to follow one exponential. This is a proposed low-order statistical model, not a complete acoustic simulator or a new decay law.

A room encoder estimates the coefficients from a context window. Hold them fixed over an utterance in the initial static-room experiment. A speech encoder updates $q$ rapidly. The resulting reverberation state conditions the existing complex filter together with ordinary speech features. Power estimates cannot be subtracted directly from an audio waveform; phase-sensitive reconstruction stays in the complex-filter path.

The coefficients should initially be constrained away from unstable or excessively long memories. Stable coefficients make the isolated recurrence stable for bounded input; they do not prove stability of an arbitrary network that feeds its output back into its input. Start with a feed-forward speech-power estimate to avoid that extra complication.

#The candidate contribution

The research hypothesis is that separating relatively persistent room memory from rapidly changing speech content improves generalization and speech preservation at a fixed computation budget. The physical state is active during inference, rather than appearing only as a training penalty. That is a design distinction to investigate, not sufficient proof of novelty.

A generic recurrent network can also learn memory. Therefore, the decisive comparison is a matched recurrent module with the same state size, context and training data but without the source-driven decay structure. Another necessary control supplies estimated reverberation time as an ordinary feature. If those controls work equally well, the proposed physical parameterization has not demonstrated additional value.

#Why this is substantive work for both researchers

Mino defines the speech target, chooses the backbone, analyses speech distortion and evaluates quality and recognition. Vũ derives the statistical approximation, parameter constraints and identifiability limits, and studies how errors in the estimated decay affect recovery. Together, the task is to design and test a physics-inspired speech model—not to append physical terminology to an otherwise unchanged network.

#4. What the literature already establishes

The following sources were checked in this refinement. Methods/results sections were examined where noted; abstract-level sources are used only to establish overlap.

• IF-CorrNet already uses inter-frame correlations for monaural dereverberation. It is a direct baseline for the representation and filtering mechanism, not merely related work. The earlier project review examined its methods and ablations. Paper.
• Speech dereverberation constrained on room impulse response characteristics introduces a physical loss based on frequency-dependent energy decay. Its physical blocks are discarded at inference. Its reported quality and signal-distortion metrics do not improve uniformly, which is a useful warning against assuming a more physical loss necessarily improves every speech endpoint. Method and results examined. Bahrman et al., 2024.
• BUDDy already combines a speech diffusion prior with blind room-response estimation. Jointly estimating speech and room properties is therefore established. Its primary paper was opened and screened; a detailed implementation audit remains. BUDDy, 2024.
• VINP combines a neural speech prior with a probabilistic convolutive model to estimate speech and the room response. Its formulation and conclusion were examined. A proposed neural acoustic model must be compared with this broader family, rather than only with an unconstrained network. VINP, 2025.
• WPE and neural-supported WPE already protect short-term speech correlations using delayed prediction. An adaptive predictor or a prediction delay alone is not new. Neural-supported dereverberation study, 2023.

There is substantial older work on statistical late-reverberation power prediction as well. The recurrence above should be treated as a starting baseline formulation. A focused review of recursive reverberation estimators, learned state-space models and neural WPE remains necessary before claiming a new architecture. Current novelty confidence is provisional.

#5. Spatial extension: combine echo memory with microphone evidence

With multiple microphones, the model gains a second clue: how signals vary across space. Under an ideal homogeneous three-dimensional diffuse field with omnidirectional microphones, the expected coherence between two microphones separated by distance $d_{ij}$ has the form

$$\Gamma_{ij}(f)=\frac{\sin(2\pi f d_{ij}/c)}{2\pi f d_{ij}/c},$$

with the value at zero defined by continuity. Here, $c$ is sound speed. This is an idealized prior, not a universal description of room noise or echoes. Coherence-based dereverberation is established; the primary source was screened at abstract level in this pass. Coherent-to-diffuse power ratio estimation.

A possible extension combines the temporal reverberation-power estimate with a learned spatial covariance model. Temporal evidence asks whether energy can be explained by earlier speech; spatial evidence asks whether its microphone pattern resembles the assumed reverberant field. Neither clue alone identifies the target speaker. A competing speaker is also a physical sound source and may be spatially coherent.

The model should allow deviations from ideal diffuseness. Covariance matrices must remain positive semidefinite, meaning that they represent nonnegative power for every linear combination of microphones. A structured positive-semidefinite parameterization is preferable to an arbitrary matrix correction. Compare an ideal diffuse model, a fully learned covariance and a constrained mixture of the two.

This extension is not automatically novel: non-diffuse reverberant covariance estimation and neural-guided convolutional beamforming already exist. Reverberant covariance estimation, 2020; neural-guided joint denoising, dereverberation and separation, 2021. The proposed question is whether the explicit coupling to speech-driven decay improves robustness or data efficiency. These sources were screened through primary abstracts, not fully reproduced.

Start with the temporal module. Adding spatial decomposition, graph attention, uncertainty estimation and self-supervision simultaneously would make it difficult to identify what contributed.

#6. A pilot that can disprove the idea

Begin with single-speaker reverberant speech and additive noise. Fix the output as direct-plus-early speech using a declared early-response window relative to direct arrival. This is an enhancement/dereverberation experiment; two-speaker separation is a later extension with additional source-assignment ambiguity.

Generate different utterances through the same room impulse response and the same utterance through different responses. Split speakers and rooms independently. This allows a controlled check of whether the slow representation captures room behaviour rather than speaker identity. Do not require reverberation waveforms from different utterances to be identical; only the relevant room-dependent coefficients should be shared.

Compare:

All neural variants should receive the same observation context and output filter support. Count the context needed by the room encoder as latency if it uses future audio. Use a static-utterance offline pilot first; do not label it streaming prematurely.

Test strong and weak reverberation, noisy and quiet recordings, speech onsets, sustained vowels, pauses and already-clean speech. After the controlled test, add measured room responses and then movement or abrupt acoustic changes. Measured-response convolution remains different from live simultaneous recording.

Report signal recovery against the declared target, appropriate quality/intelligibility metrics, word error rate with a fixed recognizer, runtime and added latency. Physical-parameter accuracy is diagnostic, not the primary success criterion. Estimated decay coefficients need not be uniquely identifiable from a short speech recording.

• Reproduce a small fixed-backbone baseline with a declared target and context.
• Test oracle decay information to establish whether useful improvement is possible.
• Compare learned physical memory against matched recurrence and simple room conditioning.
• Check speech preservation and transfer to held-out rooms with multiple training seeds.
• Add spatial evidence only if the temporal mechanism demonstrates a defensible benefit.

Reject or simplify the method if gains vanish under matched controls, if estimated room states mainly track the speaker, or if echo reduction worsens speech preservation. An oracle gain followed by failure of estimated coefficients identifies an estimation problem; no oracle gain suggests the chosen conditioning mechanism has little headroom.

#7. Current recommendation and limits

The leading candidate is speech-preserving enhancement with an explicit learned reverberation memory, followed by a temporal–spatial extension if justified. The topic remains recognizably speech research, while Vũ has a substantial role in deriving and testing the physical model.

The broad collaboration direction is now clear; the exact architecture is not yet selected by the owner. This document records a concrete hypothesis and the evidence needed to keep or reject it. No experiments, dataset downloads or compute commitments have been made. The next research step is a focused nearest-neighbour literature audit and a small diagnostic pilot—not a claim that this formulation is already novel.