wav2vec2-xls-r-parlaspeech-hr

This model for Croatian ASR is based on the [facebook/wav2vec2-xls-r-300m model](https://huggingface.co/facebook/wav2vec2-xls-r-300m) and was fine-tuned with 300 hours of recordings and transcripts from the ASR Croatian parliament dataset [ParlaSpeech-HR v1.0](http://hdl.handle.net/11356/1494). If you use this model, please cite the following paper: Nikola Ljubešić, Danijel Koržinek, Peter Rupnik, Ivo-Pavao Jazbec. ParlaSpeech-HR -- a freely available ASR dataset for Croatian bootstrapped from the ParlaMint corpus. http://www.lrec-conf.org/proceedings/lrec2022/workshops/ParlaCLARINIII/pdf/2022.parlaclariniii-1.16.pdf Evaluation is performed on the dev and test portions of the [ParlaSpeech-HR v1.0](http://hdl.handle.net/11356/1494) dataset. There are multiple models available, and in terms of CER and WER, the best-performing model is [wav2vec2-large-slavic-parlaspeech-hr-lm](https://huggingface.co/classla/wav2vec2-large-slavic-parlaspeech-hr-lm). from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC import soundfile as sf import torch import os device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") processor = Wav2Vec2Processor.from_pretrained( "classla/wav2vec2-xls-r-parlaspeech-hr") model = Wav2Vec2ForCTC.from_pretrained("classla/wav2vec2-xls-r-parlaspeech-hr") os.system("wget https://huggingface.co/classla/wav2vec2-xls-r-parlaspeech-hr/raw/main/00020570a.flac.wav") speech, sample_rate = sf.read("00020570a.flac.wav") input_values = processor(speech, sampling_rate=sample_rate, return_tensors="pt").input_values.to(device) os.system("rm 00020570a.flac.wav") logits = model.to(device)(input_values).logits predicted_ids = torch.argmax(logits, dim=-1) transcription = processor.decode(predicted_ids[0]).lower() In fine-tuning, the following arguments were used: