During my Go SSH adventures at Hacking with Go I wanted to write a simple SSH harvester. As usual, the tool turned out to be much larger than I thought.
I realized I cannot find any examples of SSH certificate verification. There are a few examples for host keys here and there. Even the certs_test.go
file just checks the host name. There was a typo in an error message1 in the crypto/ssh
package but I think because this is not very much used, had gone unreported.
Here's my step by step guide to writing this tool by piggybacking on SSH host verification callbacks. Hopefully this will make it easier for the next person.
You can find the code here:
TL;DR: verifying SSH servers
- Create an instance of ssh.CertChecker.
- Set callback functions for
IsHostAuthority
,IsRevoked
and optionallyHostKeyFallback
.IsHostAuthority
's callback should returntrue
for valid certificates.IsRevoked
's callback should returnfalse
for valid certificates.HostKeyFallback
's callback should returnnil
for valid certificates.
- Create an instance of ssh.ClientConfig.
- Set
HostKeyCallback
inClientConfig
to&ssh.CertChecker.CheckHostKey
. - CheckHostKey will verify the certificate based on other callback functions.
- The certificate can be accessed in
IsRevoked
callback function.
Go to Parsing SSH certificates
to skip the fodder.
Table of Contents
- Table of Contents
- Before we start
- Code analysis
- Parsing SSH certificates <-- This is the important part
- SSH Harvester in action
- Conclusion
Before we start
- Think of this as a simple Proof of Concept (PoC). I will keep this version in the clone. However, I will keep building upon this to make it a full-blown SSH vuln scanner using non-standard libraries.
- I kept to standard libraries. For example I know there are better CLI managers than flag out there like Cobra and CLI.
- Everything is in one big file, this will hopefully be fixed in the vuln scanner.
Code analysis
I am not completely trying to deflect criticism but security scripts are a different beast. You want to write something that does some specific thing and alerts you the moment it stops working so you can fix/redo. That said, please let me know if there are any huge errors or if I can do something much better.
Constants and usage
We can either pass a file with -in
. The file should have one address on each line:
|
|
Or we can pass addresses with -t
separated by commas:
SSHHarvester.exe -t 127.0.0.1:22,[2001:db8::68]:1234
Output file is specified with -out
.
|
|
This is pretty standard. You might want to change the default username/password. Ultimately we do not care about logging in, we just want to connect and get host info.
Init function
We setup flags, logging and check flags. flag
package does not have mutually_exclusive_group
from Python's Argparse
package. It needs to be done manually. I will most likely move to a community cli package after this.
|
|
errorExit
just calls logger.Fatalf
with a message. Logging the message and returning from main with status code 1.
|
|
Custom flag type
We are using a custom flag type for -t
. This allows us to pass multiple addresses separated by ,
and get a slice of addresses directly. This is done through implementing the flag.value which contains two methods String()
and Set()
. In simple words:
- Create a new type
mytype
. - Create two methods with
*mytype
receivers namedString()
andSet()
.String()
casts the custom type to astring
and returns it.Set(string)
has astring
argument and populates the type, returns an error if applicable.
- Create a new flag without an initial value:
- Call
flag.NewFlagSet(&var,
instead offlag.String(
. - Call
flag.Var(
instead offlag.StringVar(
orflag.IntVar(
.
- Call
I have written more about the flag
package in Hacking with Go - 03.1.
|
|
SSHServer struct
We use a struct and some methods to hold server info. The SSHServer
struct has these fields:
|
|
Not all fields will be populated. For example Hostname
and PublicKey
are only populated if the server responds with a public key. If it has a cert, then Cert
will be populated instead.
New *SSHServer
s are created by NewSSHServer
.
|
|
net.SplitHostPort
splits host:port
into two strings but it does not check the validity of either part. Meaning you can pass 500.500.500.500:70000
and it will be accepted because the format is correct.
To check if the IP is valid, we can use net.ParseIP
and check the result (it's nil
if it was not parsed correctly). However, we do not know if we are dealing with hostnames like example.com:1234
. But we can check if ports are in the correct range.
SSHServers struct
SSHServers
is a slice of SSHServer
pointers. It has a Stringer method (a String
method that returns a string representation of receiver).
|
|
Struct to JSON
ToJSON
converts a struct to a JSON string. If the second argument is true
, it pretty prints it by indenting.
|
|
This is one of the useful things I learned while working on this code. It's a pretty cool way of converting structs into strings. When printing with "%+v"
format string, field pointers are not dereferenced and it will print the memory address. However, marshalling to JSON dereferences every field.
Note: When JSON-ing structs, make sure to mark fields as exportable by starting their names with capital letters. The JSON package cannot see them otherwise.
Utilities
There are a couple of misc functions.
readTargetFile
reads addresses from a file (one address on each line) and returns a []string
.
writeReport
gets a slice of SSHServer
s (SSHServers
to be exact), converts it to string (the Stringer we saw earlier will try to convert it to JSON first) and writes it to a file. The final file will be a JSON object that can be parsed.
Parsing SSH certificates <-- This is the important part
Inside ssh.ClientConfig there's a callback HostKeyCallback
. This function should return nil
if host is verified. Read Phil Pennock's blogpost Golang SSH Security for the history behind it.
Let's expand the tl;dr steps:
Step 1: Create ssh.CertChecker
We are interested in the following three ssh.CertChecker fields. All of them are callback functions:
|
|
Don't worry about the functions for now. But remember these callback functions are only required to have a specific return value but can have any number of arguments. This is very useful we can pass our SSHServer
objects and populate them inside these functions.
Step 2: Set Callback functions
Set callback functions for these three fields.
IsHostAuthority
IsHostAuthority
must be defined. If not, we get a run-time error:
golang.org/x/crypto/ssh.(*CertChecker).CheckHostKey(0xc04206a140, 0xc0420080c0,
0xc, 0x68d700, 0xc042058450, 0x68df80, 0xc0420a2000, 0x1, 0x8)
Z:/Go/src/golang.org/x/crypto/ssh/certs.go:301 +0xae
golang.org/x/crypto/ssh.(*CertChecker).CheckHostKey-fm(0xc0420080c0, 0xc,
0x68d700, 0xc042058450, 0x68df80, 0xc0420a2000, 0x0, 0x0)
Z:/Go/src/hackingwithgo/04.5-01-ssh-harvester.go:205 +0x70
...
To discover the error cause, one must look at the source code for CheckHostKey. We'll see that CheckHostKey
calls IsHostAuthority
.
|
|
So what does this function do?
First it tries to get a certificate from key PublicKey
(by casting). If the cast is not successful, it uses HostKeyFallBack
to verity server's public key instead.
Then the function checks if the certificate type is HostCert
. SSH differentiates between host and client certificates. For example OpenSSH's keygen
uses the -h
switch to sign and create a host key.
Another of our callbacks, IsHostAuthority
is called next. If it returns false
, the certificate is not valid. The docs say:
// IsHostAuthority should report whether the key is recognized as
// an authority for this host. This allows for certificates to be
// signed by other keys, and for those other keys to only be valid
// signers for particular hostnames. This must be set if this
// CertChecker will be checking host certificates.
This is just fancy talk for verifying the CA and performing certificate pinning. In other words we can check:
- Is the certificate signed by a valid CA? Note, unlike TLS certs, most SSH certs are signed by internal CAs. Often we are relying on a hardcoded CA for verification.
- Is the certificate signed by the valid CA? We don't want certs signed by other CAs.
net.SplitHostPort
(we already used it above) splits host:port
into host
and port
and passes hostname
to CheckCert
.
CheckCert
does a couple of more checks. Most notably it calls another one of our functions IsRevoked
.
|
|
IsHostAuthority callback
Not every function can be a callback function. Each function needs to return certain type. IsHostAuthority
requires the callback function to have this return type:
func(ssh.PublicKey, string) bool
In other words, our callback function needs to return a function of that type.
First we create a custom type (it's not defined in the package) and then create a function that returns that type:
|
|
If we want the connection to continue, the internal function needs to return true
.
IsRevoked
IsRevoked
is not mandatory. If it's not set, it's ignored. Meaning there's no automatic certificate revocation checks happening without it. Note the typo in the error message: . The typo has now been corrected. Honestly, I think this just means programs do not use this function (or I am terribly wrong and am using something which should not be used). If certificate is valid, this function must return certicate
nil
or false
.
IsRevoked callback
For the goal of grabbing the certificate and processing it, IsRevoked
is the most useful. It gets the certificate as a parameter and we can do parse or verify it inside the function. IsRevoked
must return:
func(cert *Certificate) bool
Again we define that function type and declare our own function:
|
|
Inside IsRevoked
we have access to the SSH certificate. Here we just assign it to the Cert
field.
If you want to verify the certificate, this is the place.
Question!!!! Solved
Help me if you can. I don't like returning unnamed functions like this. But unless I create global variables, I need to be able to access s *SSHServer
inside certCallback
to populate it. The function type is strict so I cannot add arguments.
I think defining the inside function as a method will work. Am I write? Wrong? Please let me know if you know the answer.
Method is the way to go or just use anonymous functions. I don't like them but there's nothing wrong with using one.
HostKeyFallback
Not all servers have SSH certificates. In fact, most servers probably do not. If server does not send a certificate, this function will be called (and the connection will terminate if this function is not defined).
If server is valid this function should return nil.
|
|
Here we grab server's public key and hostname.
With these three callbacks set, we can move to the next step.
Step 3: Create ssh.ClientConfig
ssh.ClientConfig is needed for every SSH connection in Go. You can read about creating SSH connections in Hacking with Go - 04.4.
|
|
Timeout
is also important. we do not want goroutines to wait forever connecting to inaccessible addresses. It's set to 5 seconds by default. Can be changed in the constants.
Banner callback
Banner callback is another important function for information gathering. By now, you know the drill.
|
|
We store the banner message and return nil
. Any other return value will terminate the connection.
Step 4: ClientConfig.HostKeyCallback
This callback starts the server verification chain. It needs a function with ssh.HostKeyCallback type:
type HostKeyCallback func(hostname string, remote net.Addr, key PublicKey) error
The package actually suggests (*CertChecker) CheckHostKey (we looked at its source code earlier). Looking inside ClientConfig
, you can see I am passing it like this:
HostKeyCallback: certCheck.CheckHostKey,
This is where everything clicks. We created a certCheck
and set its callback functions. Now we are passing it to be called when we connect to a server.
Other ways of verifying servers
If you do not want to verify server's certificate, you can plug in three different types of functions here.
ssh.FixedHostKey(key PublicKey)
: Returns a function to check the hostkey.ssh.InsecureIgnoreHostKey()
: Ignore everything! Danger! Will Robinson!Custom host verifier
: Return nil if host is ok, otherwise return an error.
Read more about them in the verifying host.
A note about ssh.InsecureIgnoreHostKey()
After the breaking change as a consequence of the Golang SSH security blog post linked earlier, everyone seems to be using this. I am not in the position to tell you how to write your code. But make sure you know what you are doing when using this function. cough hashicorp packer cough.
Step 5: Connecting to SSH servers
Here comes the concurrent part. We have a list of addresses and our callbacks are set correctly. Time to connect to servers with discover
.
discover method
|
|
First we defer releasing the waitgroup and the log message. This waitgroup will be explained later. In short, it's here to ensure that all discover
goroutines are finished before starting the next stage.
Next are CertCheck
and ClientConfig
. We have already seen them. And finally we are connecting with ssh.Dial
.
Goroutines and sync.WaitGroups
Each connection is done in its own goroutine. This means, we have to wait for these to complete before processing the results. We use sync.WaitGroups
. For a longer version please read Hacking with Go - 02.6 - Syncing goroutines. But a tl;dr description is:
- Every time a goroutine is started, we add one to the waitgroup (note we need to do this in the calling function, not inside the goroutine).
- When the goroutine returns we subtract one (the
defer discoveryWG.Done()
indiscover
). - Wait in main for all goroutines to finish with
discoveryWG.Wait()
. This will block the program until they all return.
SSH Harvester in action
And finally we can see the tool in action.
If the server returns a certificate:
If it returns a public key, HostKeyFallBack
is triggered and we can it:
Note, server's have different keys for different ciphersuits. For example dsa
, ecdsa
, rsa
and ed25519
(the DJB curve). Depending what ciphersuite client supports, you may see one of these. That's another TODO.
Conclusion
It took me a couple of days to figure everything out because I could not find any examples or tutorials. But now we know how to verify SSH certificates. Hope this is useful, if you have any feedback please let me know.
I should have actually sent a patch. But signing up for Gerrit was a pain. Would have been the easiest way to become a "Golang contributor" and put it in my Twitter bio/resume (kidding). ↩︎