Writing a client with Twisted Conch¶
In the original days of computing, rsh/rlogin were used to connect to remote computers and execute commands. These commands had the problem that the passwords and commands were sent in the clear. To solve this problem, the SSH protocol was created. Twisted Conch implements the second version of this protocol.
Using an SSH Command Endpoint¶
If your objective is to execute a command on a remote host over an SSH connection, then the easiest approach may be to use twisted.conch.endpoints.SSHCommandClientEndpoint . If you haven’t used endpoints before, first take a look at the endpoint howto to get an idea of how endpoints work in general.
Conch provides an endpoint implementation which establishes an SSH connection, performs necessary authentication, opens a channel, and launches a command in that channel. It then associates the output of that command with the input of a protocol you supply, and associates output from that protocol with the input of that command. Effectively, this lets you ignore most of the complexity of SSH and just interact with a remote process as though it were any other stream-oriented connection - such as TCP or SSL.
Conch also provides an endpoint that is initialized with an already established SSH connection. This endpoint just opens a new channel on the existing connection and launches a command in that.
SSHCommandClientEndpoint is about as simple as using any
other stream-oriented client endpoint. Just create the endpoint defining
where the SSH server to connect to is and a factory defining what kind of
protocol to use to interact with the command and let them get to work
using the endpoint’s
#!/usr/bin/env python # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. if __name__ == '__main__': import sys import echoclient_ssh from twisted.internet.task import react react(echoclient_ssh.main, sys.argv[1:]) import os, getpass from twisted.python.filepath import FilePath from twisted.python.usage import Options from twisted.internet.defer import Deferred from twisted.internet.protocol import Factory, Protocol from twisted.internet.endpoints import UNIXClientEndpoint from twisted.conch.ssh.keys import EncryptedKeyError, Key from twisted.conch.client.knownhosts import KnownHostsFile from twisted.conch.endpoints import SSHCommandClientEndpoint class EchoOptions(Options): optParameters = [ ("host", "h", "localhost", "hostname of the SSH server to which to connect"), ("port", "p", 22, "port number of SSH server to which to connect", int), ("username", "u", getpass.getuser(), "username with which to authenticate with the SSH server"), ("identity", "i", None, "file from which to read a private key to use for authentication"), ("password", None, None, "password to use for authentication"), ("knownhosts", "k", "~/.ssh/known_hosts", "file containing known ssh server public key data"), ] optFlags = [ ["no-agent", None, "Disable use of key agent"], ] class NoiseProtocol(Protocol): def connectionMade(self): self.finished = Deferred() self.strings = ["bif", "pow", "zot"] self.sendNoise() def sendNoise(self): if self.strings: self.transport.write(self.strings.pop(0) + "\n") else: self.transport.loseConnection() def dataReceived(self, data): print "Server says:", data self.sendNoise() def connectionLost(self, reason): self.finished.callback(None) def readKey(path): try: return Key.fromFile(path) except EncryptedKeyError: passphrase = getpass.getpass("%r keyphrase: " % (path,)) return Key.fromFile(path, passphrase=passphrase) class ConnectionParameters(object): def __init__(self, reactor, host, port, username, password, keys, knownHosts, agent): self.reactor = reactor self.host = host self.port = port self.username = username self.password = password self.keys = keys self.knownHosts = knownHosts self.agent = agent @classmethod def fromCommandLine(cls, reactor, argv): config = EchoOptions() config.parseOptions(argv) keys =  if config["identity"]: keyPath = os.path.expanduser(config["identity"]) if os.path.exists(keyPath): keys.append(readKey(keyPath)) knownHostsPath = FilePath(os.path.expanduser(config["knownhosts"])) if knownHostsPath.exists(): knownHosts = KnownHostsFile.fromPath(knownHostsPath) else: knownHosts = None if config["no-agent"] or "SSH_AUTH_SOCK" not in os.environ: agentEndpoint = None else: agentEndpoint = UNIXClientEndpoint( reactor, os.environ["SSH_AUTH_SOCK"]) return cls( reactor, config["host"], config["port"], config["username"], config["password"], keys, knownHosts, agentEndpoint) def endpointForCommand(self, command): return SSHCommandClientEndpoint.newConnection( self.reactor, command, self.username, self.host, port=self.port, keys=self.keys, password=self.password, agentEndpoint=self.agent, knownHosts=self.knownHosts) def main(reactor, *argv): parameters = ConnectionParameters.fromCommandLine(reactor, argv) endpoint = parameters.endpointForCommand(b"/bin/cat") factory = Factory() factory.protocol = NoiseProtocol d = endpoint.connect(factory) d.addCallback(lambda proto: proto.finished) return d
For completeness, this example includes a lot of code to support different
styles of authentication, reading (and possibly updating) existing
known_hosts files, and parsing command line options. Focus on
the latter half of the
main function to see the code that is
most directly responsible for actually doing the necessary SSH connection
SSHCommandClientEndpoint accepts quite a few options, since
there is a lot of flexibility in SSH and many possible different server
configurations, but once the endpoint object itself is created, its use is
no more complicated than the use of any other endpoint: pass a factory to
connect method and attach a callback to the
Deferred to do something with the protocol
instance. If you use an endpoint that creates new connections, the connection
attempt can be cancelled by calling
cancel() on this
In this case, the connected protocol instance is only used to make the
example wait until the client has finished talking to the server, which
happens after the small amount of example data has been sent to the server
and bounced back by the
/bin/cat process the
protocol is interacting with.
Several of the options accepted by
SSHCommandClientEndpoint.newConnection should be easy to understand.
The endpoint takes a reactor which it uses to do any and all I/O it needs to do.
It also takes a command which it executes on the remote server once the SSH connection is established and authenticated; this command is a single string, perhaps including spaces or other special shell symbols, and is interpreted by a shell on the server.
It takes a username with which it identifies itself to the server for authentication purposes.
It takes an optional password argument which will also be used for authentication - if the server supports password authentication (prefer keys instead where possible, see below).
It takes a host (either a name or an IP address) and a port number, defining where to connect.
Some of the other options may bear further explanation.
keys argument gives any SSH Key objects which may be useful for authentication.
These keys are available to the endpoint for authentication, but only keys that the server indicates are useful will actually be used.
This argument is optional.
If key authentication against the server is either unnecessary or undesired, it may be omitted entirely.
agentEndpoint argument gives the
SSHCommandClientEndpoint an opportunity to connect to an SSH authentication agent.
The agent may already be loaded with keys, or may have some other way to authenticate a connection.
Using the agent can mean the process actually establishing the SSH connection doesn’t need to load any authentication material (passwords or keys) itself (often convenient in case keys are encrypted and potentially more secure, since only the agent process ever actually holds the secrets).
The value for this argument is another
Often in a typical NIX desktop environment, the *SSH_AUTH_SOCK environment variable will give the location of a AF_UNIX socket.
This explains the value
echoclient_ssh.py assigns this parameter when –no-agent is not given.
knownHosts argument accepts a KnownHostsFile instance and controls how server keys are checked and stored.
This object has the opportunity to reject server keys if they differ from expectations.
It can also save server keys when they are first observed.
Finally, there is one option that is not demonstrated in the example - the
This argument is closely related to the
knownHosts argument described above.
KnownHostsFile may require user-input under certain circumstances - for example, to ask if it should accept a server key the first time it is observed.
ui object is how this user-input is obtained.
By default, a ConsoleUI instance associated with /dev/tty will be used.
This gives about the same behavior as is seen in a standard command-line ssh client.
See SSHCommandClientEndpoint.newConnection for details about how edge cases are handled for this default value.
For use of
SSHCommandClientEndpoint that is intended to be completely autonomous, applications will probably want to specify a custom
ui object which can make the necessary decisions without user-input.
It is also possible to run commands (one or more) over an
already-established connection. This is done using the alternate
#!/usr/bin/env python # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. if __name__ == '__main__': import sys import echoclient_shared_ssh from twisted.internet.task import react react(echoclient_shared_ssh.main, sys.argv[1:]) from twisted.internet.task import cooperate from twisted.internet.defer import Deferred, gatherResults from twisted.internet.protocol import Factory, Protocol from twisted.conch.endpoints import SSHCommandClientEndpoint from echoclient_ssh import ConnectionParameters class PrinterProtocol(Protocol): def dataReceived(self, data): print "Got some data:", data, def connectionLost(self, reason): print "Lost my connection" self.factory.done.callback(None) def main(reactor, *argv): parameters = ConnectionParameters.fromCommandLine(reactor, argv) endpoint = parameters.endpointForCommand(b"/bin/cat") done =  factory = Factory() factory.protocol = Protocol d = endpoint.connect(factory) def gotConnection(proto): conn = proto.transport.conn for i in range(50): factory = Factory() factory.protocol = PrinterProtocol factory.done = Deferred() done.append(factory.done) e = SSHCommandClientEndpoint.existingConnection( conn, b"/bin/echo %d" % (i,)) yield e.connect(factory) d.addCallback(gotConnection) d.addCallback(lambda work: cooperate(work).whenDone()) d.addCallback(lambda ignored: gatherResults(done)) return d
Writing a client¶
In case the endpoint is missing some necessary functionality, or in case you want to interact with a different part of an SSH server - such as one of its subsystems (for example, SFTP), you may need to use the lower-level Conch client interface. This is described below.
Writing a client with Conch involves sub-classing 4 classes: twisted.conch.ssh.transport.SSHClientTransport , twisted.conch.ssh.userauth.SSHUserAuthClient , twisted.conch.ssh.connection.SSHConnection , and twisted.conch.ssh.channel.SSHChannel . We’ll start out
SSHClientTransport because it’s the base
of the client.
from twisted.conch import error from twisted.conch.ssh import transport from twisted.internet import defer class ClientTransport(transport.SSHClientTransport): def verifyHostKey(self, pubKey, fingerprint): if fingerprint != 'b1:94:6a:c9:24:92:d2:34:7c:62:35:b4:d2:61:11:84': return defer.fail(error.ConchError('bad key')) else: return defer.succeed(1) def connectionSecure(self): self.requestService(ClientUserAuth('user', ClientConnection()))
See how easy it is?
handles the negotiation of encryption and the verification of keys
for you. The one security element that you as a client writer need to
verifyHostKey() . This method
is called with two strings: the public key sent by the server and its
fingerprint. You should verify the host key the server sends, either
by checking against a hard-coded value as in the example, or by asking
verifyHostKey returns a twisted.internet.defer.Deferred which gets a callback
if the host key is valid, or an errback if it is not. Note that in the
above, replace ‘user’ with the username you’re attempting to ssh with,
for instance a call to
os.getlogin() for the
The second method you need to implement is
connectionSecure() . It is called when the
encryption is set up and other services can be run. The example requests
ClientUserAuth service be started.
This service will be discussed next.
from twisted.conch.ssh import connection class ClientConnection(connection.SSHConnection): def serviceStarted(self): self.openChannel(CatChannel(conn = self))
SSHConnection is the easiest,
as it’s only responsible for starting the channels. It has
other methods, those will be examined when we look at
from twisted.conch.ssh import channel, common class CatChannel(channel.SSHChannel): name = 'session' def channelOpen(self, data): d = self.conn.sendRequest(self, 'exec', common.NS('cat'), wantReply = 1) d.addCallback(self._cbSendRequest) self.catData = '' def _cbSendRequest(self, ignored): self.write('This data will be echoed back to us by "cat."\r\n') self.conn.sendEOF(self) self.loseConnection() def dataReceived(self, data): self.catData += data def closed(self): print 'We got this from "cat":', self.catData
Now that we’ve spent all this time getting the server and
client connected, here is where that work pays off.
SSHChannel is the interface between you and the
other side. This particular channel opens a session and plays with the
‘cat’ program, but your channel can implement anything, so long as the
server supports it.
channelOpen() method is
where everything gets started. It gets passed a chunk of data;
however, this chunk is usually nothing and can be ignored.
channelOpen() initializes our
channel, and sends a request to the other side, using the
sendRequest() method of the
SSHConnection object. Requests are used to send
events to the other side. We pass the method self so that it knows to
send the request for this channel. The 2nd argument of ‘exec’ tells the
server that we want to execute a command. The third argument is the data
that accompanies the request. common.NS encodes
the data as a length-prefixed string, which is how the server expects
the data. We also say that we want a reply saying that the process has a
sendRequest() then returns a
Deferred which we add a callback for.
Once the callback fires, we send the data.
SSHChannel supports the twisted.internet.interfaces.ITransport
it can be given to Protocols to run them over the secure
connection. In our case, we just write the data directly.
sendEOF() does not follow the interface,
but Conch uses it to tell the other side that we will write no
down our side of the connection, but we will still receive data
dataReceived() . The
closed() method is called when both sides of the
connection are closed, and we use it to display the data we received
(which should be the same as the data we sent.)
Finally, let’s actually invoke the code we’ve set up.
The main() function¶
from twisted.internet import protocol, reactor def main(): factory = protocol.ClientFactory() factory.protocol = ClientTransport reactor.connectTCP('localhost', 22, factory) reactor.run() if __name__ == "__main__": main()
connectTCP() to connect to
localhost, port 22 (the standard port for ssh), and pass it an instance
of twisted.internet.protocol.ClientFactory .
This instance has the attribute
set to our earlier
class. Note that the protocol attribute is set to the class
ClientTransport , not an instance of
ClientTransport ! When the
connectTCP call completes, the protocol will be
called to create a
- this then invokes all our previous work.
It’s worth noting that in the example
reactor.run() call never returns.
If you want to make the program exit, call
reactor.stop() in the earlier
If you wish to observe the interactions in more detail, adding a call
reactor.run() call will send all
logging to stdout.