[Question] Sending audio from a different app to gnome-calls

ethical_haquer · September 15, 2024, 3:07pm

Hello!
I’m experimenting with using an app’s output audio as a source for call audio, instead of the mic.

The device I’m using is a PinephonePro running PMOS (postmarketOS) Edge.

I have been able to send audio from an app (so far I’ve tested with Amberol) to Sound Recorder no problem, with two different methods:

Destroy the existing links for Amberol and Sound Recorder, and then create links between them:

# Delete existing Sound Recorder links
pw-link -l -I | grep "Sound Recorder" | grep "|->" | awk '{print $1}' | xargs -n 1 pw-link -d
# Delete existing Amberol links
pw-link -l -I | grep "Amberol" | grep "|<-" | awk '{print $1}' | xargs -n 1 pw-link -d
# Link Amberol's FL to Sound Recorder's FL
pw-link -L Amberol:output_FL "Sound Recorder:input_FL"
# Link Amberol's FR to Sound Recorder's FR
pw-link -L Amberol:output_FR "Sound Recorder:input_FR"

Create a virtual mic (from here):

pw-loopback --capture-props='node.target=Amberol' --playback-props='media.class=Audio/Source node.name=virtmic node.description="VirtualMic"'

This method is pretty neat, as I can then just select “VirtualMic” as the input device in settings, open up Sound Recorder, and it will record the music instead of the mic.

The problem I’m having is, whereas these above methods work for Sound Recorder (and other apps), they don’t work for gnome-calls.

When a phone call is started, the input will get switched to the real mic.

If I select “Virtual Mic” as the input during the call, it doesn’t affect the call, which continues to use the real mic.

I also don’t understand why gnome-calls doesn’t show up in qpwgraph like other apps (see screenshot below).

Is it possible to make gnome-calls receive audio from another app?

If so, how?

Let me know if you have any questions! TIA.

Screenshot of qpwgraph

See how Amberol and Sound Recorder are shown? gnome-calls never shows up in qpwgraph.

FeRDNYC · September 15, 2024, 10:44pm

Most likely because, while /usr/bin/gnome-calls is linked with EVERYTHING, on Fedora (including tons of both Wayland and Xorg libraries), it isn’t linked with a single audio library. No codecs, no stream interfaces, nothin’.

What it IS linked with is the separate libcallaudio-0.1.so.0 library, which is part of the separate callaudiod project. Its dependencies are (…wait for it…):

libasound2-dev
libglib2.0-dev
libpulse-dev

…libasound2 is better known as the ALSA client library, and if it’s directly manipulating ALSA devices then it’s definitely not on PipeWire’s radar. (Also, it doesn’t sound like it actually streams the audio at all, so much as it just… configures the backend device routing.)

So if it’s connecting, say, your microphone’s input to your speaker output… directly in ALSA… then there’s nothing to (hi)jack in a patchbay. And definitely nothing that would make gnome-calls show up, since all of that’s being handled by a completely separate callaudiod process.

(callaudiod has a companion CLI tool, BTW:)

$ callaudiocli --help
Usage:
  callaudiocli [OPTION?] - A helper tool for callaudiod

Help Options:
  -h, --help               Show help options

Application Options:
  -m, --select-mode        Select mode
  -s, --enable-speaker     Enable speaker
  -u, --mute-mic           Mute microphone
  -S, --status             Print status

$ callaudiocli -S
Selected mode: CALL_AUDIO_MODE_DEFAULT
Speaker enabled: CALL_AUDIO_SPEAKER_OFF
Mic muted: CALL_AUDIO_MIC_OFF

ethical_haquer · September 16, 2024, 12:58am

Thanks for the reply!
I’ll start looking into doing it with ALSA.

ethical_haquer · September 18, 2024, 11:17pm

I tried doing the stuff mentioned on this page, but I wasn’t able to make any progress. I’m still learning about how the audio system works as far as PipeWire and ALSA coexisting on a system. If anyone has any info, please let me know!

FeRDNYC · October 25, 2024, 12:10am

Well, turns out I steered you wrong there; while callaudiod does indeed link with the ALSA library, it’s only because PulseAudio devices are mapped to ALSA hardware devices, and will therefore contain references to ALSA device information that callaudiod wants access to. But it does fully support PipeWire (well, in its PulseAudio-emulating form, as it’s still using the PulseAudio library to manage PipeWire devices).

The real issue is that callaudiod is extremely particular about what devices it will interact with. There are numerous reports of it not finding devices, or automagically selecting the wrong device, or configuring it in the wrong way.

But there’s also the reason that it’s particular, which is that (as I earlier surmised, but have now better confirmed from reading more of the code) call audio in a device like a Pinephone isn’t routed through Linux at all. It’s directly wired into the hardware.

When a call begins, callaudiod ensures (among other things) that the person on the other end of the call can hear your voice. This is done by: Unmuting the microphone on the default hardware device.

That’s it. That’s all that’s done, and all that NEEDS to be done, because the Pinephone’s “telephone” hardware is physically wired to the microphone built into the phone. Nothing is streamed or connected in PipeWire because, just like in a non-smart telephone, the microphone is simply one of the physical hardware components of the phone itself.

The difference in a smartphone is that the microphone is also accessible from within Linux. But that doesn’t mean that call audio is routed through Linux. It’s not. (AFAICT, without having access to one of those devices.) The Pinephone’s actual-phone functionality appears to be implemented at the hardware level; it’s not a “softphone” that would stream audio through Linux devices in order to do capture/playback during a call.

To route audio from a different device into a phone call, you’d need an audio sink that represents the “audio in” for the telephony circuitry — the one the microphone is normally connected to for the call. But there isn’t any such sink, because that side of things ISN’T exposed to Linux. The audio capture device for a phone call is the microphone, and only the microphone, because it’s the (audio-input) device that’s physically connected to the telephony hardware.

ethical_haquer · October 29, 2024, 10:08pm

Thanks for the info!
I’ll have to have a go at it again, maybe (hopefully) there’s still a way to get audio from an app to the modem. I’ll post again when I’ve tried some more.

ethical_haquer · October 29, 2024, 10:16pm

I just stumbled across this page: “Audio on PinePhone”. Here’s a quote from it:

The diagram shows multiple different paths, but only a few are improtant for connecting audio from the mic1 to the modem (bb) and from modem to the earphone on the top of the phone.
The remaining 4 routes are meant for the CPU to be able to record me, record the other caller, play back audio to me, and to play back audio to the other caller. Each of these routes can be used separately by sending/reading audio data to/from the left (me) and the right (other caller) channels of the playback and capture PCM devices. (see above)

I still have a lot to read, but I think it’s possible. Let me know if you find anything else interesting. Thanks again!

FeRDNYC · November 5, 2024, 7:01pm

I mean… anything’s possible, I guess. That quote is describing a hardware diagram… specifically, this one:

Which, if it’s labeled accurately, seems to show that the microphone and headphone interfaces are mono paths on a stereo interface, with the Left channel connected to the actual hardware (mic-in to modem, for example, or call audio out from modem to the earpiece/headphones — though that gets split to both channels, on its way to the hardware). The Right channels then hold the paths you’re talking about, like the “playback to modem”.

So the key may be that “L/R source select” notation that all of those audio interfaces have — if you want to play audio out to the modem, instead of having it capture the microphone, the key may not be about routing any audio paths differently, but rather to mute the Left channel on the modem audio interface and unmute the Right channel.

If they can be enabled together, you may even be able to unmute the Right channel and have playback audio sent to the modem, while still being able to talk over the mic. Otherwise, you may need to toggle those “AIF2 Capture Switches” to switch sources, which would (as noted) be done in alsa.

While this is true:

You can dump the list of all controls using alsactl store -f - . You can also set these controls up using alsamixer or by modifying and loading the text file generated by alsactl via alsactl restore -f <path> .

It may be easier to use amixer to manipulate the interfaces, since you can modify them interactively instead of having to edit and load configuration files. (Once you’ve figured out the correct configuration, then you can dump that config to make it loadable.)

amixer -c 0 contents will spit out all of the controls for card 0 in a compact format that gives you useful information like whether a given control is writable or not. For example, the motherboard audio interface on my desktop looks like this:

$ amixer -c 0 controls
numid=26,iface=CARD,name='Front Headphone Jack'
  ; type=BOOLEAN,access=r-------,values=1
  : values=off
numid=24,iface=CARD,name='Line Jack'
  ; type=BOOLEAN,access=r-------,values=1
  : values=off
numid=25,iface=CARD,name='Line Out Jack'
  ; type=BOOLEAN,access=r-------,values=1
  : values=off
numid=23,iface=CARD,name='Mic Jack'
  ; type=BOOLEAN,access=r-------,values=1
  : values=off
numid=27,iface=CARD,name='Speaker Phantom Jack'
  ; type=BOOLEAN,access=r-------,values=1
  : values=on
numid=22,iface=MIXER,name='Master Playback Switch'
  ; type=BOOLEAN,access=rw------,values=1
  : values=on
numid=21,iface=MIXER,name='Master Playback Volume'
  ; type=INTEGER,access=rw---R--,values=1,min=0,max=64,step=0
  : values=41
  | dBscale-min=-64.00dB,step=1.00dB,mute=0
numid=4,iface=MIXER,name='Headphone Playback Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=3,iface=MIXER,name='Headphone Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=64,step=0
  : values=0,0
  | dBscale-min=-64.00dB,step=1.00dB,mute=0
numid=31,iface=MIXER,name='PCM Playback Volume'
  ; type=INTEGER,access=rw---RW-,values=2,min=0,max=255,step=0
  : values=251,251
  | dBscale-min=-51.00dB,step=0.20dB,mute=0
numid=20,iface=MIXER,name='Line Boost Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=3,step=0
  : values=0,0
  | dBscale-min=0.00dB,step=10.00dB,mute=0
numid=2,iface=MIXER,name='Line Out Playback Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=1,iface=MIXER,name='Line Out Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=64,step=0
  : values=0,0
  | dBscale-min=-64.00dB,step=1.00dB,mute=0
numid=11,iface=MIXER,name='Line Playback Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=10,iface=MIXER,name='Line Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=31,step=0
  : values=0,0
  | dBscale-min=-34.50dB,step=1.50dB,mute=0
numid=19,iface=MIXER,name='Mic Boost Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=3,step=0
  : values=0,0
  | dBscale-min=0.00dB,step=10.00dB,mute=0
numid=9,iface=MIXER,name='Mic Playback Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=8,iface=MIXER,name='Mic Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=31,step=0
  : values=0,0
  | dBscale-min=-34.50dB,step=1.50dB,mute=0
numid=16,iface=MIXER,name='Capture Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=18,iface=MIXER,name='Capture Switch',index=1
  ; type=BOOLEAN,access=rw------,values=2
  : values=off,off
numid=15,iface=MIXER,name='Capture Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=31,step=0
  : values=0,0
  | dBscale-min=-13.50dB,step=1.50dB,mute=0
numid=17,iface=MIXER,name='Capture Volume',index=1
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=31,step=0
  : values=0,0
  | dBscale-min=-13.50dB,step=1.50dB,mute=0
numid=7,iface=MIXER,name='Loopback Mixing'
  ; type=ENUMERATED,access=rw------,values=1,items=2
  ; Item #0 'Disabled'
  ; Item #1 'Enabled'
  : values=0
numid=12,iface=MIXER,name='Auto-Mute Mode'
  ; type=ENUMERATED,access=rw------,values=1,items=3
  ; Item #0 'Disabled'
  ; Item #1 'Speaker Only'
  ; Item #2 'Line Out+Speaker'
  : values=0
numid=32,iface=MIXER,name='Digital Capture Volume'
  ; type=INTEGER,access=rw---RW-,values=2,min=0,max=120,step=0
  : values=60,60
  | dBscale-min=-30.00dB,step=0.50dB,mute=0
numid=13,iface=MIXER,name='Input Source'
  ; type=ENUMERATED,access=rw------,values=1,items=2
  ; Item #0 'Mic'
  ; Item #1 'Line'
  : values=0
numid=14,iface=MIXER,name='Input Source',index=1
  ; type=ENUMERATED,access=rw------,values=1,items=2
  ; Item #0 'Mic'
  ; Item #1 'Line'
  : values=0
numid=6,iface=MIXER,name='Speaker Playback Switch'
  ; type=BOOLEAN,access=rw------,values=2
  : values=on,on
numid=5,iface=MIXER,name='Speaker Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=64,step=0
  : values=64,64
  | dBscale-min=-64.00dB,step=1.00dB,mute=0
numid=29,iface=PCM,name='Capture Channel Map'
  ; type=INTEGER,access=r--v-R--,values=2,min=0,max=36,step=0
  : values=0,0
  | container
    | chmap-fixed=FL,FR
numid=28,iface=PCM,name='Playback Channel Map'
  ; type=INTEGER,access=r--v-R--,values=2,min=0,max=36,step=0
  : values=0,0
  | container
    | chmap-fixed=FL,FR
numid=30,iface=PCM,name='Capture Channel Map',device=2
  ; type=INTEGER,access=r--v-R--,values=2,min=0,max=36,step=0
  : values=0,0
  | container
    | chmap-fixed=FL,FR

So I can see the status of 5 different jacks (read-only), I have control over the Master Playback, Headphone Playback, PCM Playback, Line Out Playback, Line Playback, and Mic Playback Volumes, plus a switch to activate/deactivate each. There are Line and Microphone Boost Volume controls, selectors for things like Loopback Mixing, Auto-Mute Mode, and Input Sources, plus capture volume controls. Finally, some channel mappings are presented — again, read-only, as they’re there for informational purposes only.

To change the value of a writable control, you can use amixer -c 0 cset <id> <value>. The <id> can be any sufficiently unique part of the first line of each control listed — so, amixer -c 0 cset iface=MIXER,name='Master Playback Volume' 50% and amixer -c 0 cset numid=21 32 would both set my card’s master volume to 50% (as its raw max is 64).

ethical_haquer · December 11, 2024, 6:32pm

@FeRDNYC
I’ll have to start looking into this again, thanks for the info!