Objective

As described here:

https://unix.stackexchange.com/questions/388519/bash-wait-for-process-in-process-substitution-even-if-command-is-invalid

Parent does not wait for child process inside process substitution. wait command is also ignored.

Due to this, for example, following can occur:

1
2
3
4
5
6

prompt$ seq 1 5 > >(cat)
prompt$ 1
2
3
4
5

What’s happening here is that, prompt is returning before child process cat completes outputting all sequences from 1 to 5.

This article shows some example of implementing workaround, to wait for child.

Process substitution shines for bash logging

In bash, you can prepend message to stdout and stderr by doing below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

#!/bin/bash
# sbst.sh

echo_info() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ INFO: $@";
}

echo_error() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ ERROR: $@" 1>&2
}

prepend_info() { while IFS='' read -r line; do echo_info "$line"; done; };
prepend_err() { while IFS='' read -r line; do echo_error "$line"; done; };

exec > >(prepend_info)
exec 2> >(prepend_err)

exec > >(prepend_info) in this line, stdout is redirecting output to function prepend_info.
prepend_info then calls echo_info, outputting the passed stdout message with some additional info before it.

Similar thing is happening with exec 2> >(prepend_err), but with stderr.

Below those code, I further add following codes

1
2

echo Hello World
2> echo Redirecting this msg to stderr

Following is the otuput I get when executed

1
2
3
4
5

[root@host bin]# bash sbst.sh
[root@host bin]# 2020-10-05T06:32:41+0000 117310 ERROR: Redirecting this msg to stderr
2020-10-05T06:32:41+0000 117310 INFO: Hello World

[root@host bin]#

We now have a script which adds time and pid in from of stdout and stderr message:
2020-10-05T06:32:41+0000 117310 INFO: Hello World
2020-10-05T06:32:41+0000 117310 ERROR: Redirecting this msg to stderr

However notice that there’s two issues from in above result

1. Prompt is waiting for user input before outputting all messages
2. Order of echo is out of order

(1) is due to aforementioned “process substitution not waiting for child”. In the next section, I have implemented the workaround.
(2) is due to multiprocessing. Essentially the second echo is completing more faster than the first echo. It will be difficult to avoid and will not be covered in this article.

Implementing workaround

Implementing workaround is fairly simple.

You just have to wrap all your commands with { var_name=$( { and } 3>&1 >&4 4>&-); } 4>&1.

Here’s my full script after implementing the workaround.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/bin/bash

{ proc_sbst=$( {

echo_info() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ INFO: $@";
}

echo_error() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ ERROR: $@" 1>&2
}

prepend_info() { while IFS='' read -r line; do echo_info "$line"; done; };
prepend_err() { while IFS='' read -r line; do echo_error "$line"; done; };

exec > >(prepend_info)
exec 2> >(prepend_err)

echo Hello World
>&2 echo Redirecting this msg to stderr

} 3>&1 >&4 4>&-); } 4>&1

When I run this, prompt now waits for parent and child process to finish.

1
2
3
4

[root@host bin]# bash sbst.sh
2020-10-05T07:13:02+0000 117640 INFO: Hello World
2020-10-05T07:13:02+0000 117640 ERROR: Redirecting this msg to stderr
[root@host bin]#

What if I want to output to stdout and stderr without extra info

execs that forward output to function are declared to capture fd 1 (default stdout) and fd 2 (default stderr).
This means, you can avoid it by outputting to different fd.

This fd can be file, as shown in first example.

Or this fd can be a copy of stdout and stderr as shown in second example.

Example 1: Outputting stdout/stderr to file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

echo Hello World > /tmp/stdout.log
bad_command 2> /tmp/stderr.log

...

[root@host bin]# bash sbst.sh
[root@host bin]# cat /tmp/std*
sbst.sh: line 45: bad_command: command not found
Hello World
[root@host bin]#

Example 2: Making copied fd of stdout and stderr

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/bin/bash

{ proc_sbst=$( {

exec 8>&1
exec 9>&2

echo_info() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ INFO: $@";
}

echo_error() {
    echo $(date '+%Y-%m-%dT%H:%M:%S%z') "$$ ERROR: $@" 1>&2
}

prepend_info() { while IFS='' read -r line; do echo_info "$line"; done; };
prepend_err() { while IFS='' read -r line; do echo_error "$line"; done; };

exec > >(prepend_info)
exec 2> >(prepend_err)

echo Hello World >&8
bad_command 2>&9

} 3>&1 >&4 4>&-); } 4>&1

...

[root@host bin]# bash sbst.sh
Hello World
sbst.sh: line 45: bad_command: command not found
[root@host bin]#

Notice the exec 8>&1 and exec 9>&2 at the top of the script. This is making copy of fd 1 (stdout) and fd 2 (stderr).
You will redirect to fd 8 for stdout and fd 9 for stderr.