Single channel speech denoising by DDPG reinforcement learning agent
<p dir="ltr">Speech denoising (SD) covers the algorithms that suppress the background noise from the contaminated speech and improve its clarity. In this paper, a novel SD algorithm is presented based on the deep deterministic policy gradient (DDPG) agent; an off-policy reinforcement...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | |
| منشور في: |
2025
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513531482734592 |
|---|---|
| author | Sania Gul (18272227) |
| author2 | Muhammad Salman Khan (7202543) |
| author2_role | author |
| author_facet | Sania Gul (18272227) Muhammad Salman Khan (7202543) |
| author_role | author |
| dc.creator.none.fl_str_mv | Sania Gul (18272227) Muhammad Salman Khan (7202543) |
| dc.date.none.fl_str_mv | 2025-07-19T15:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.apacoust.2025.110954 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Single_channel_speech_denoising_by_DDPG_reinforcement_learning_agent/30971164 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Data management and data science Machine learning Speech denoising Reinforcement learning DDPG agentTD3 agent Reward function Time–frequency masking |
| dc.title.none.fl_str_mv | Single channel speech denoising by DDPG reinforcement learning agent |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Speech denoising (SD) covers the algorithms that suppress the background noise from the contaminated speech and improve its clarity. In this paper, a novel SD algorithm is presented based on the deep deterministic policy gradient (DDPG) agent; an off-policy reinforcement learning (RL) agent with a continuous action space. The noisy speech is first converted from the time domain to the time–frequency (TF) domain by taking its short-time Fourier transform (STFT), and then two separate DDPG agents are trained on the magnitude and phase components of the STFT. The reward function used for training these agents is the relative perceptual quality score of speech. After training, the DDPG agents generate the magnitude and phase soft masks, when noisy speech is given as input to them. These masks are then applied to the complex STFT matrix of the noisy speech to obtain the denoised speech. For matched testing data, the proposed system offers an improvement of 1.55 points in the perceptual evaluation of speech quality (PESQ) over the unprocessed speech, the highest among the other recent state-of-the-art models used for comparison in this paper. It achieves this performance by utilizing data that is 7 times smaller than that required by other models. Also, its learnable parameters are the lowest among all models, almost 12 times less than the next most compact model, based on another continuous RL agent (policy gradient (PG)) for estimating its convolutional kernels. When cascaded with a coloured spectrogram-based SD model, the proposed model further improves the PESQ by 0.07, CSIG by 1.23, and COVL by 1.4 points; the metrics estimating respectively the perceived quality of speech, its composite distortion, and its composite overall quality, surpassing all other baseline systems compared here. In the cascaded configuration, our proposed model offers the highest gain in PESQ (by 0.78 points) at 50 times fewer episodes, compared to the already trained speech enhancement and recognition models utilizing discrete agents for their performance improvement.</p><h2 dir="ltr">Other Information</h2><p dir="ltr">Published in: Applied Acoustics<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.apacoust.2025.110954" target="_blank">https://dx.doi.org/10.1016/j.apacoust.2025.110954</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_1f7409d3a9ae9ced80369443e1f35ecf |
| identifier_str_mv | 10.1016/j.apacoust.2025.110954 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/30971164 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Single channel speech denoising by DDPG reinforcement learning agentSania Gul (18272227)Muhammad Salman Khan (7202543)Information and computing sciencesArtificial intelligenceData management and data scienceMachine learningSpeech denoisingReinforcement learningDDPG agentTD3 agentReward functionTime–frequency masking<p dir="ltr">Speech denoising (SD) covers the algorithms that suppress the background noise from the contaminated speech and improve its clarity. In this paper, a novel SD algorithm is presented based on the deep deterministic policy gradient (DDPG) agent; an off-policy reinforcement learning (RL) agent with a continuous action space. The noisy speech is first converted from the time domain to the time–frequency (TF) domain by taking its short-time Fourier transform (STFT), and then two separate DDPG agents are trained on the magnitude and phase components of the STFT. The reward function used for training these agents is the relative perceptual quality score of speech. After training, the DDPG agents generate the magnitude and phase soft masks, when noisy speech is given as input to them. These masks are then applied to the complex STFT matrix of the noisy speech to obtain the denoised speech. For matched testing data, the proposed system offers an improvement of 1.55 points in the perceptual evaluation of speech quality (PESQ) over the unprocessed speech, the highest among the other recent state-of-the-art models used for comparison in this paper. It achieves this performance by utilizing data that is 7 times smaller than that required by other models. Also, its learnable parameters are the lowest among all models, almost 12 times less than the next most compact model, based on another continuous RL agent (policy gradient (PG)) for estimating its convolutional kernels. When cascaded with a coloured spectrogram-based SD model, the proposed model further improves the PESQ by 0.07, CSIG by 1.23, and COVL by 1.4 points; the metrics estimating respectively the perceived quality of speech, its composite distortion, and its composite overall quality, surpassing all other baseline systems compared here. In the cascaded configuration, our proposed model offers the highest gain in PESQ (by 0.78 points) at 50 times fewer episodes, compared to the already trained speech enhancement and recognition models utilizing discrete agents for their performance improvement.</p><h2 dir="ltr">Other Information</h2><p dir="ltr">Published in: Applied Acoustics<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.apacoust.2025.110954" target="_blank">https://dx.doi.org/10.1016/j.apacoust.2025.110954</a></p>2025-07-19T15:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.apacoust.2025.110954https://figshare.com/articles/journal_contribution/Single_channel_speech_denoising_by_DDPG_reinforcement_learning_agent/30971164CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/309711642025-07-19T15:00:00Z |
| spellingShingle | Single channel speech denoising by DDPG reinforcement learning agent Sania Gul (18272227) Information and computing sciences Artificial intelligence Data management and data science Machine learning Speech denoising Reinforcement learning DDPG agentTD3 agent Reward function Time–frequency masking |
| status_str | publishedVersion |
| title | Single channel speech denoising by DDPG reinforcement learning agent |
| title_full | Single channel speech denoising by DDPG reinforcement learning agent |
| title_fullStr | Single channel speech denoising by DDPG reinforcement learning agent |
| title_full_unstemmed | Single channel speech denoising by DDPG reinforcement learning agent |
| title_short | Single channel speech denoising by DDPG reinforcement learning agent |
| title_sort | Single channel speech denoising by DDPG reinforcement learning agent |
| topic | Information and computing sciences Artificial intelligence Data management and data science Machine learning Speech denoising Reinforcement learning DDPG agentTD3 agent Reward function Time–frequency masking |