ROS Sound
have to git clone https://github.com/ros-drivers/audio_common.git on kinetic since it isn’t a package there yet.
https://github.com/ros-drivers/audio_common
sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
It looks very limited- only uses default microphone, can’t spawn multiple copies and set each to a different microphone. Is that a gstreamer limitation, or could it be trivially added to the ros audio code?
gst-launch-1.0 alsasrc ! audioconvert ! audioresample ! alsasink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
Redistribute latency...
Sounds like it works, has delay of a good fraction of a second.
audio common builds and runs fine in kinetic (will get errors about gstappsink.h missing if the plugins dev is not installed).
selecting a specific microphone
https://community.nxp.com/thread/332384
Found it. I had to use alsasrc
gst-launch alsasrc device=hw:2 ! audioconvert ! audioresample ! alawenc ! rtppcmapay ! udpsink host=192.168.20.26 port=5001
So need to get same functionality into audio_capture.cpp
Plug in a second microphone (on top of the built in laptop mic):
card 1: PCH [HDA Intel PCH], device 0: ALC3226 Analog [ALC3226 Analog]
Subdevices: 0/1
Subdevice #0: subdevice #0
card 2: C930e [Logitech Webcam C930e], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
This works
roslaunch audio_capture capture.launch device:="hw:1,0"
but this doesn’t
roslaunch audio_capture capture.launch device:="hw:2,0"
...
[ERROR] [/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/audio_capture/src/audio_capture.cpp]:[174] [gstreamer: Could not open audio device for recording.]
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
what(): boost: mutex lock failed in pthread_mutex_lock: Invalid argument
See if pure gst works:
gst-launch-1.0 alsasrc device=hw:2 ! audioconvert ! audioresample ! alsasink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
ERROR: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Could not get/set settings from/on resource.
Additional debug info:
gstalsasrc.c(465): set_hwparams (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Rate doesn't match (requested 44100Hz, get 0Hz)
Execution ended after 0:00:00.000474618
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Need to set permissions somewhere?
arecord -D hw:2,0 -c 2 -f S16_LE -r 16000 -d 5 /tmp/test.wav
This successfully records from the webcam microphone.
I probably need to duplicate the S16_LE and 16000 rate to get audio_capture to work.
This works:
gst-launch-1.0 alsasrc device=hw:2 ! audioconvert ! audioresample ! 'audio/x-raw, format=S16LE, rate=16000, channels=2' ! alsasink
Do I need a resample inbetween the convert and encoder? No, a filter.
This works:
roslaunch audio_capture capture.launch device:=""
but this doesn’t:
roslaunch audio_capture capture.launch device:="hw:1"
...
ERROR] [/audio/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/audio_capture/src/audio_capture.cpp]:[187] [gstreamer: Internal data flow error.]
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
what(): boost: mutex lock failed in pthread_mutex_lock: Invalid argument
and this works:
roslaunch audio_capture capture.launch device:="hw:2"
So can use either microphone now, but the hw:1 or hw:1,0 option should work.
$ arecord -D hw:1,0 -d 5 /tmp/test.wav
Recording WAVE '/tmp/test.wav' : Unsigned 8 bit, Rate 8000 Hz, Mono
arecord: set_params:1233: Sample format non available
Available formats:
- S16_LE
- S32_LE
These settings work:
arecord -D hw:1,0 -f S16_LE -c 2 -r 44100 -d 5 /tmp/test.wav
pacmd list-sources
...
* index: 2
name: <alsa_input.pci-0000_00_1b.0.analog-stereo>
driver: <module-alsa-card.c>
flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
state: SUSPENDED
suspend cause: IDLE
priority: 9959
volume: front-left: 8251 / 13% / -54.00 dB, front-right: 8251 / 13% / -54.00 dB
balance 0.00
base volume: 5206 / 8% / -66.00 dB
volume steps: 65537
muted: no
current latency: 0.00 ms
max rewind: 0 KiB
sample spec: s16le 2ch 44100Hz
channel map: front-left,front-right
Stereo
used by: 0
linked by: 0
configured latency: 0.00 ms; range is 0.50 .. 371.52 ms
card: 1 <alsa_card.pci-0000_00_1b.0>
module: 7
properties:
alsa.resolution_bits = "16"
device.api = "alsa"
device.class = "sound"
alsa.class = "generic"
alsa.subclass = "generic-mix"
alsa.name = "ALC3226 Analog"
alsa.id = "ALC3226 Analog"
alsa.subdevice = "0"
alsa.subdevice_name = "subdevice #0"
alsa.device = "0"
alsa.card = "1"
alsa.card_name = "HDA Intel PCH"
alsa.long_card_name = "HDA Intel PCH at 0xf7d30000 irq 33"
alsa.driver_name = "snd_hda_intel"
device.bus_path = "pci-0000:00:1b.0"
sysfs.path = "/devices/pci0000:00/0000:00:1b.0/sound/card1"
device.bus = "pci"
device.vendor.id = "8086"
device.vendor.name = "Intel Corporation"
device.product.id = "8c20"
device.product.name = "8 Series/C220 Series Chipset High Definition Audio Controller"
device.form_factor = "internal"
device.string = "front:1"
device.buffering.buffer_size = "65536"
device.buffering.fragment_size = "32768"
device.access_mode = "mmap+timer"
device.profile.name = "analog-stereo"
device.profile.description = "Analog Stereo"
device.description = "Built-in Audio Analog Stereo"
alsa.mixer_name = "Realtek ALC3226"
alsa.components = "HDA:10ec0292,10280684,00100001"
module-udev-detect.discovered = "1"
device.icon_name = "audio-card-pci"
ports:
analog-input-internal-mic: Internal Microphone (priority 8900, latency offset 0 usec, available: unknown)
properties:
device.icon_name = "audio-input-microphone"
analog-input-dock-mic: Dock Microphone (priority 7800, latency offset 0 usec, available: no)
properties:
device.icon_name = "audio-input-microphone"
analog-input-headset-mic: Headset Microphone (priority 8700, latency offset 0 usec, available: unknown)
properties:
device.icon_name = "audio-input-microphone"
active port: <analog-input-internal-mic>
So setting the sample rate works:
roslaunch audio_capture capture.launch device:="hw:1" sample_rate:=44100
But why does device:=”” work? Are the setting auto-detected then- why can’t I auto-detect for any device? Or is there something magic about the default device that automatically filters/resamples/etc. as necessary?
Wave
Does the wave sound audio message just have a raw waveform?
roslaunch audio_capture capture.launch device:="hw:1" sample_rate:=44100 format:="mp3"
rostopic hz /audio/audio
38.227
Then cut and past a single message from /audio/audio, the array is 522 entries long.
38.227 * 522 = 20000 samples/second?
roslaunch audio_capture capture.launch device:="hw:1" sample_rate:=44100 format:="wave"
(audio_capture:6789): GStreamer-WARNING : 0.10-style raw audio caps are being created. Should be audio/x-raw,format=(string).. now.
[ERROR] [/audio/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/audio_capture/src/audio_capture.cpp]:[132] [Unsupported media type.]
x-raw-int -> x-raw in audio_capture.cpp
roslaunch audio_capture capture.launch device:="hw:1" sample_rate:=44100 format:="wave" channels:=2
100 Hz audio messages 1764 uint8s per message But every other entry is zero- the stereo isn’t actually working?
1764 * 100 / 2 / 2 = 44100
The second division by 2 is for converting each uint8 to a 16 bit number.
Maybe the second number that is always zero is low order bits, only really getting 8-bit audio not 16-bit?
->
Could I make a decoder node that doesn’t play a sound out of the speaker, but just converts an mp3 encoded message to wave?
Would be great to be able to have a ros vu meter.
Are there generic 1D signal analysis, filtering, and visualization tools already, just need to convert to right format?