When I look at journalctl, it tells me the PID and the program name(or service name?) of a log entry.

Then I wondered, logs are created by other processes, how do systemd-journald know the PID of these processes when processes may only write raw strings to the unix domain socket which systemd-journald is listenning. Also, do sytemd-journald always use the same technique to detect the PID of a piece of log data even when processes are producing log using functions like sd_journal_sendv()?

Is there any documentation I should read about this?

I read JdeBP's answer and know systemd-journald listen on an Unix Domian Socket, but even if can know the peer socket address who send the log message, how does it know the PID? What if that sending socket is opened by many non-parent-children processes?

It receives the pid via the SCM_CREDENTIALS ancillary data on the unix socket with recvmsg(), see unix(7). The credentials don't have to be sent explicitly.

Example:

$ cc -Wall scm_cred.c -o scm_cred

$ ./scm_cred

scm_cred: received from 10114: pid=10114 uid=2000 gid=2000

Processes with CAP_SYS_ADMIN data can send whatever pid they want via SCM_CREDENTIALS; in the case of systemd-journald, this means they can fake entries as if logged by another process:

# cc -Wall fake.c -o fake

# setcap CAP_SYS_ADMIN+ep fake



$ ./fake `pgrep -f /usr/sbin/sshd`



# journalctl --no-pager -n 1

...

Dec 29 11:04:57 debin sshd[419]: fake log message from 14202

# rm fake

# lsb_release -d

Description:    Debian GNU/Linux 9.6 (stretch)

The systemd code which handles datagrams and credentials sent via ancillary data in in the server_process_datagram() function from journald-server.c. The syslog(3) library function will send log data via a SOCK_DGRAM socket by default. Neither systemd-journald nor rsyslogd accept SOCK_STREAM connections on /dev/log.

scm_cred.c

#define _GNU_SOURCE     1

#include <sys/socket.h>

#include <sys/un.h>

#include <unistd.h>

#include <err.h>



int main(int ac, char **av){

        int fd[2]; pid_t pid;

        if(socketpair(AF_LOCAL, SOCK_DGRAM, 0, fd)) err(1, "socketpair");

        if((pid = fork()) == -1) err(1, "fork");

        if(pid){ /* parent */

                int on = 1;

                union {

                        struct cmsghdr h;

                        char data[CMSG_SPACE(sizeof(struct ucred))];

                } buf;

                struct msghdr m = {0};

                struct ucred *uc = (struct ucred*)CMSG_DATA(&buf.h);

                m.msg_control = &buf;

                m.msg_controllen = sizeof buf;

                if(setsockopt(fd[0], SOL_SOCKET, SO_PASSCRED, &on, sizeof on))

                        err(1, "setsockopt");

                if(recvmsg(fd[0], &m, 0) == -1)

                        err(1, "recvmsg");

                warnx("received from %d: pid=%d uid=%d gid=%d", pid,

                        uc->pid, uc->uid, uc->gid);

        }else   /* child */

                write(fd[1], "foo", 3);

        return 0;

}

fake.c

#define _GNU_SOURCE     1

#include <sys/socket.h>

#include <sys/un.h>

#include <unistd.h>

#include <stdlib.h>

#include <stdio.h>

#include <err.h>



int main(int ac, char **av){

        union {

                struct cmsghdr h;

                char data[CMSG_SPACE(sizeof(struct ucred))];

        } cm;

        int fd; char buf[256];

        struct ucred *uc = (struct ucred*)CMSG_DATA(&cm.h);

        struct msghdr m = {0};

        struct sockaddr_un ua = {AF_UNIX, "/dev/log"};

        struct iovec iov = {buf};

        if((fd = socket(AF_LOCAL, SOCK_DGRAM, 0)) == -1) err(1, "socket");

        if(connect(fd, (struct sockaddr*)&ua, SUN_LEN(&ua))) err(1, "connect");

        m.msg_control = &cm;

        m.msg_controllen = cm.h.cmsg_len = CMSG_LEN(sizeof(struct ucred));

        cm.h.cmsg_level = SOL_SOCKET;

        cm.h.cmsg_type = SCM_CREDENTIALS;

        uc->pid = ac > 1 ? atoi(av[1]) : getpid();

        uc->uid = ac > 2 ? atoi(av[2]) : geteuid();

        uc->gid = ac > 3 ? atoi(av[3]) : getegid();

        iov.iov_len = snprintf(buf, sizeof buf, "<13>%s from %d",

                ac > 4 ? av[4] : "fake log message", getpid());

        if(iov.iov_len >= sizeof buf) errx(1, "message too long");

        m.msg_iov = &iov;

        m.msg_iovlen = 1;

        if(sendmsg(fd, &m, 0) == -1) err(1, "sendmsg");

        return 0;

}

The kernel tells it.

The EUID, EGID, and PID of the original client process that connected the AF_LOCAL stream socket at /run/systemd/journal/stdout is available from the kernel via the SO_PEERCRED socket option, which it uses. UCSPI-UNIX tools obtain this same information via the same system call.

Child service processes of course inherit their standard I/O file descriptors already opened (unless the parent service process changes this, of course), and so to systemd-journald all log output has the credentials of the original parent process.

Log output generated via the AF_LOCAL socket at /run/systemd/journal/socket that speaks the idiosyncratic systemd-journald protocol is coming over a datagram socket, rather than a stream one. This socket is flagged using the SO_PASSCRED socket option so that the kernel records the same information in each datagram sent, which is pulled out of each datagram by systemd-journald.

Further reading

• 
  getsockopt(). Linux Programmers' Manual. 2017-09-15.


• 
  socket. Linux Programmers' Manual. 2018-02-02.


• Jonathan de Boyne Pollard (2017). local-stream-socket-accept. nosh Guide. Softwares.


• Jonathan de Boyne Pollard (2015). "Environment variables". The gen on the UNIX Client-Server Program Interface. Frequently Given Answers.