Tag Archives: paramiko

Paramiko sftp hanging with connections between machines on the same interface of a filtering pfsense box

Odd problem; I had the following set up:

[[machine with paramiko 10.100.x.x]] –|

                                                              | —–(int X) 10.100.x.x [[pfsense]] (int Y) 10.2.x.x —– | —— [[10.2.x.x machine B]]

[[machine A 10.100.x.x]] ——————|


I had a script on the paramiko machine connecting via ssh and sftp to machines A and B.  Connections to machine B had no problem whatsoever.  Connections to machine A, however, would work 5% of the time, and drop the rest of the time either when setting up the channel to execute a command over ssh, or when invoking the sftp subsystem on the remote machine.  Normal ssh and sftp connections (not using paramiko) had no problems whatsoever.  Also, when pfSense filtering was turned off, there were also no problems.

It turned out that pfsense was dropping a lot of packets sent by paramiko due to fragmentation (logs show TCP:PA, TCP:RA and TCP:A).  Unfortunately, tweaking pfsense settings didn’t help here (some people have reported that setting Firewall Optimization Options (under Advanced > Firewall/NAT) to conservative worked – that didn’t help me unfortunately – or disabling firewall scrub worked – which I couldn’t do as it’s required by NAT).

I haven’t been able to figure out exactly what the problem is.  The packets received by machine B and machine A (with filtering off) look exactly the same.  I’m tempted to think this is a pfsense problem, although I have no specific proof (I’ve tested with multiple machines in position of machine A by the way, compared ssh settings, ensured there were no other connectivity problems in the way).

In the end, I’ve set up another network (virtual one, since these are VMs – 10.100.x.x machines plus pfsense on one physical host, and 10.2.x.x on another) connecting these VMs directly to eachother, to bypass pfsense for these connections.

Paramiko channel hangs

When sending a command via ssh using paramiko, the script would hang. eg:

def ssh_connect(self):
    """ connects to the remote server using paramiko """
    ssh = paramiko.SSHClient()
    ssh.connect(self.hostname, self.remoteport, self.remoteuser, None, None, self.keypath)
    return ssh

def ssh_command(self, command):
    """ executes long command on remote server """
    ssh = self.ssh_connect()
    channel = ssh.invoke_shell()
    stdin = channel.makefile('wb')
    stdout = channel.makefile('rb')
    ssh_out = stdout.read()
    stdout.close(); stdin.close(); ssh.close()
    return ssh_out

command = """
    tar -zxvf {rp} || echo 'deploy-copy-untar-error {rp}'
    rm {rp} || echo 'deploy-copy-delete-error {rp}'
    echo 'deploy-copy-success {rp}'
    """.format(rp = remotepath)
ssh_out = self.ssh_command(command)

You need to add ‘exit’ to the end of the command so the channel quits and the script continues. Like so:
command = “””
tar -zxvf {rp} || echo ‘deploy-copy-untar-error {rp}’
rm {rp} || echo ‘deploy-copy-delete-error {rp}’
echo ‘deploy-copy-success {rp}’
“””.format(rp = remotepath)
sshout = self.sshcommand(command)

Paramiko on Ubuntu 10.04: GMP or MPIR library not found

Getting the following error when trying to install paramiko on Ubuntu 10.04 (python 2.6.5) with pip – because of pycrypto dependancy:

warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath

Googling this problem leads to lots of results from Mac users, not to much from Ubuntu users. I don’t have this issue on Ubuntu 11.04 (python 2.7), so I assume it’s the python version.

To fix:

apt-get install python-dev

Then try again!