Skip to content

Comments

[Audio] Remove torchaudio from spectrogram transforms#9802

Merged
anteju merged 3 commits intoNVIDIA-NeMo:mainfrom
anteju:pr/audio-transform-wout-torchaudio
Jul 23, 2024
Merged

[Audio] Remove torchaudio from spectrogram transforms#9802
anteju merged 3 commits intoNVIDIA-NeMo:mainfrom
anteju:pr/audio-transform-wout-torchaudio

Conversation

@anteju
Copy link
Collaborator

@anteju anteju commented Jul 18, 2024

What does this PR do ?

This PR removes use of torchaudio from AudioToSpectrogram and SpectrogramToAudio.
Classes using torchaudio are kept under AudioToSpectrogamTA and SpectrogramToAudioTA, similarly to FilterbankFeatures and FilterbankFeaturesTA.

Collection: audio

Changelog

  • Removed torchaudio requirement for AudioToSpectrogram and SpectrogramToAudio
  • Added unit tests to check analysis, synthesis, and reconstruction
  • Added unit tests to check the update is matching torchaudio version

Usage

  • You can potentially add a usage example below
# Prepare analysis and synthesis transforms
encoder = AudioToSpectrogram(fft_length=fft_length, hop_length=hop_length)
decoder = SpectrogramToAudio(fft_length=fft_length, hop_length=hop_length)

# Analysis transform, input shape (B, C, T)
X, _ = encoder(input=x)

# Synthesis transform, output shape (B, C, T)
y, _ = decoder(input=Y)

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

@github-actions github-actions bot added the audio label Jul 18, 2024
@anteju anteju force-pushed the pr/audio-transform-wout-torchaudio branch from 3f7fc34 to 5cdd568 Compare July 19, 2024 01:06
@anteju anteju added Run CICD and removed Run CICD labels Jul 19, 2024
@anteju anteju marked this pull request as ready for review July 19, 2024 01:15
@anteju anteju added Run CICD and removed Run CICD labels Jul 19, 2024
@anteju anteju changed the title [Draft] Spectrogram transforms without torchaudio Remove torchaudio from spectrogram transforms Jul 19, 2024
@anteju anteju requested a review from racoiaws July 19, 2024 01:16
@anteju anteju changed the title Remove torchaudio from spectrogram transforms [Audio] Remove torchaudio from spectrogram transforms Jul 19, 2024
@anteju anteju force-pushed the pr/audio-transform-wout-torchaudio branch from 5cdd568 to 3187b0a Compare July 19, 2024 05:32
@anteju anteju added Run CICD and removed Run CICD labels Jul 19, 2024
@racoiaws
Copy link
Collaborator

Overall LGTM, but needs documenting of all reshapes (e.g. via einops)

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
@anteju anteju force-pushed the pr/audio-transform-wout-torchaudio branch from dc63f1f to 8102bd0 Compare July 23, 2024 00:00
@anteju anteju added Run CICD and removed Run CICD labels Jul 23, 2024
@anteju anteju requested a review from racoiaws July 23, 2024 00:00
Signed-off-by: Ante Jukić <ajukic@nvidia.com>
@anteju anteju force-pushed the pr/audio-transform-wout-torchaudio branch from 8102bd0 to 8c606e2 Compare July 23, 2024 00:16
@anteju anteju removed the Run CICD label Jul 23, 2024
Copy link
Collaborator

@racoiaws racoiaws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anteju anteju merged commit 7e16edb into NVIDIA-NeMo:main Jul 23, 2024
tonyjie pushed a commit to tonyjie/NeMo that referenced this pull request Jul 24, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
akoumpa pushed a commit that referenced this pull request Jul 25, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
BoxiangW pushed a commit to BoxiangW/NeMo that referenced this pull request Jul 30, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Signed-off-by: Boxiang Wang <boxiangw@nvidia.com>
xuanzic pushed a commit to xuanzic/NeMo that referenced this pull request Aug 1, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Signed-off-by: Vivian Chen <xuanzic@example.com>
kchike pushed a commit to kchike/NeMo that referenced this pull request Aug 8, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Signed-off-by: kchike <kohei.chike@jp.ricoh.com>
monica-sekoyan pushed a commit that referenced this pull request Oct 14, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 5, 2024
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
XuesongYang pushed a commit to paarthneekhara/NeMo that referenced this pull request Jan 18, 2025
* Spectrogram transforms without torchaudio

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

* Use einops, added comments

Signed-off-by: Ante Jukić <ajukic@nvidia.com>

---------

Signed-off-by: Ante Jukić <ajukic@nvidia.com>
Co-authored-by: Roman Korostik <roman.korostik@pm.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants