Cfrtjryk

This is a problem I've been having for a long time, but every time I try to figure something out I get lost, so I figured I'd better ask here where maybe someone more experienced could help me.

Background

My Raspberry Pi is running Raspbian Jessie, and I use SSH often to login into it and execute commands remotely. During my first SSH sessions I noticed that an ssh-agent process was spawned on the RPi every time I logged in, but never killed when exiting: logging in and out several times caused a bunch of ssh-agent processes to be spawned just to be left hanging there doing nothing. Fiddling around and reading man pages and answers here and there I recently understood the purpose of ssh-agent, and I also learned that it should normally get killed when logging out, so I began to ask myself why it wasn't. Furthermore, I noticed that issuing source ~/.bashrc causes another instance of ssh-agent to be spawned. I read on the relative man page that the environment variable SSH_AGENT_PID should be defined because the ssh-agent program should be started within an eval to execute its output and define such variables, which are then used by other SSH-related commands, including ssh-agent -k (to kill the agent relative to the current session), so I ran echo $SSH_AGENT_PID and echo $SSH_AUTH_SOCK, but they were both empty. I suddenly realized: probably the process doesn't get killed on logout because ssh-agent -k tries to read its PID from the environment variable which is not set.

The problem

Since ssh-agent is not being killed on logout, and this for sure happens because the needed environment variables aren't set, it can only mean one thing: whoever calls ssh-agent on login probably doesn't do it in the proper way (which would be eval "$(ssh-agent -s)"). So I thought: well, what's the problem? I'll just find whichever configuration file, service, or login script gets executed to start the agent and manually fix it! Where on earth could it be?

What I've tried

Since I noticed that an ssh-agent is spawned every time I call source ~/.bashrc, this was the first file I inspected, but nothing there even remotely referenced anything related to SSH. I kept searching using vi for the string ssh inside all the following files, but found nothing:

~/.bashrc

~/.profile

/etc/bash.bashrc

/etc/profile

/etc/profile.d/ (every file in this folder)

/etc/environment

Is there some more file that could be involved in source ~/.bashrc? I don't really know.

Then I searched for relevant systemd services, but only found ssh.service, which is WantedBy=multi-user.target, and therefore is not run on login (and well, that's obvious since this is the SSH server daemon).

I also tried moving every single file in my /home/pi folder to a temporary folder and logging out and back in again, but ssh-agent still spawned.

Eventually, I also fired the last shot I had in the chamber: I ran find / -name 'ssh-agent' as root, which only printed /usr/bin/ssh-agent, an executable, so I created a fake executable which basically only logged the parent command:

#! /bin/bash

ps -o args= $PPID        > /home/pi/LOG

cat /proc/$PPID/cmdline >> /home/pi/LOG

I renamed the real /usr/bin/ssh-agent and replaced it with the fake one setting the right permissions/user/group, ran source ~/.bashrc again, then printed the LOG file:

-bash

-bash

Not a single clue on what is going on.

Some more details

I'm adding some more details, I don't know if they could be helpful or not, but you know... better safe than sorry.

• Here's my .bashrc.




• I created a new user named dummy using useradd -m dummy, and logging into it doesn't start any ssh-agent (I feel like this could mean something). The diff /home/pi/.bashrc /home/dummy/.bashrc shows basically nothing (just a comment I made), same for diff /home/pi/.profile /home/dummy/.profile.



• 
  

  The agent socket is created without problem even though SSH_AUTH_SOCK is not set:

  
  
  

  pi:~$ ls -lAh /tmp/ssh-vQRTAyj7DJry/
  
  total 0
  
  srw------- 1 pi pi 0 Jan 28 03:12 agent.1328
  
  

  
  
  

  Not sure why, but the number on the socket filename is always the one immediately before the PID of the ssh-agent process.

  
  


• 
  

  Snippet from htop:

  
  
  

   PID  USER  PRI  NI  VIRT   RES   SHR  S  Command
  
     1  root   20   0  5472  3900  2728  S  /sbin/init
  
  1329  pi     20   0  3696   224    16  S  └─ ssh-agent -s
  
  

  
  


• 
  

  Installed packages matching ssh:

  
  
  

  pi:~$ apt list --installed | grep ssh
  
  libpam-chksshpwd/oldstable,now 1.1.8-3.1+deb8u2+rpi3 armhf [installed]
  
  libssh-gcrypt-4/oldstable,now 0.6.3-4+deb8u2 armhf [installed,automatic]
  
  libssh2-1/oldstable,now 1.4.3-4.1+deb8u1 armhf [installed,automatic]
  
  openssh-client/oldstable,now 1:6.7p1-5+deb8u4 armhf [installed,automatic]
  
  openssh-server/oldstable,now 1:6.7p1-5+deb8u4 armhf [installed,automatic]
  
  openssh-sftp-server/oldstable,now 1:6.7p1-5+deb8u4 armhf [installed,automatic]
  
  ssh/oldstable,now 1:6.7p1-5+deb8u4 all [installed]
  
  sshpass/oldstable,now 1.05-1 armhf [installed]
  
  

  
  



• Searching for ssh-agent -s recursively using grep in /etc and /lib yields no results.




• I have no desktop environment installed, but I have a /etc/X11 folder with some configuration files in it. I tried renaming the folder to something else and rebooting just in case, but the process still gets spawned, so apparently that doesn't have much to do with this.

Conclusion

Now, to make it as simple as possible, I've only got two questions:

1. Where and how is this ssh-agent spawned, who issues the command?


2. Why doesn't it get called in the proper way, without setting the needed environment variables, and therefore leaving the process to hang there indefinitely?

I know a couple possible reasons:

• if you're using libpam-ssh, it can automatically start a SSH agent for you as part of a session start-up, and even automatically load your keys if they either have no passphrase or their passphrase is the same as your login password.




• if you are using gpg-agent, it can optionally perform the task of the ssh-agent too. Its shut-down is handled differently, so there will be only the SSH_AUTH_SOCK environment variable, not the SSH_AGENT_PID




• if you have SSH-agent (e.g. PuTTY's Pageant) running on your workstation and make a SSH connection with agent forwarding enabled (and the remote sshd allows it), on the remote host you will again see only SSH_AUTH_SOCK without the SSH_AGENT_PID... because the agent socket goes to sshd which tunnels it back to your local workstation's SSH agent.