Replicating request to Chef with Python RSA
Question: Goal: I need a Python 3 wrapper for Chef's REST API. Because its Python-3,
PyChef is out of the question.

Problem: I am trying to replicate the Chef request with Python RSA. But the
wrapper results in an error message: _"Invalid signature for user or client
'XXX'"_.

I approached the wrapper by trying to replicate the cURL script shown in [Chef
Authentication and Authorization with
cURL](https://docs.chef.io/auth.html#curl) using a Python RSA package: [RSA
Signing and verification](http://stuvel.eu/files/python-rsa-
doc/usage.html#signing-and-verification).

Here's my rewrite. It could be simpler but I started getting paranoid about
line breaks and headers order, so added a few unnecessary things:

    
    
    import base64
    import hashlib
    import datetime
    import rsa
    import requests
    import os
    from collections import OrderedDict
    
    body = ""
    path = "/nodes"
    client_name = "anton"
    client_key = "/Users/velvetbaldmime/.chef/anton.pem"
    # client_pub_key = "/Users/velvetbaldmime/.chef/anton.pub"
    
    hashed_body = base64.b64encode(hashlib.sha1(body.encode()).digest()).decode("ASCII")
    hashed_path = base64.b64encode(hashlib.sha1(path.encode()).digest()).decode("ASCII")
    timestamp = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
    canonical_request = 'Method:GET\\nHashed Path:{hashed_path}\\nX-Ops-Content-Hash:{hashed_body}\\nX-Ops-Timestamp:{timestamp}\\nX-Ops-UserId:{client_name}'
    canonical_request = canonical_request.format(
        hashed_body=hashed_body, hashed_path=hashed_path, timestamp=timestamp, client_name=client_name)
    headers = "X-Ops-Timestamp:{timestamp}\nX-Ops-Userid:{client_name}\nX-Chef-Version:0.10.4\nAccept:application/json\nX-Ops-Content-Hash:{hashed_body}\nX-Ops-Sign:version=1.0"
    headers = headers.format(
        hashed_body=hashed_body, hashed_path=hashed_path, timestamp=timestamp, client_name=client_name)
    
    headers = OrderedDict((a.split(":", 2)[0], a.split(":", 2)[1]) for a in headers.split("\n"))
    
    headers["X-Ops-Timestamp"] = timestamp
    
    with open(client_key, 'rb') as privatefile:
        keydata = privatefile.read()
        privkey = rsa.PrivateKey.load_pkcs1(keydata)
    
    with open("pubkey.pem", 'rb') as pubfile:
        keydata = pubfile.read()
        pubkey = rsa.PublicKey.load_pkcs1_openssl_pem(keydata)
    
    signed_request = base64.b64encode(rsa.sign(canonical_request.encode(), privkey, "SHA-1"))
    dummy_sign = base64.b64encode(rsa.sign("hello".encode(), privkey, "SHA-1"))
    
    print(dummy_sign)
    
    def chunks(l, n):
        n = max(1, n)
        return [l[i:i + n] for i in range(0, len(l), n)]
    
    auth_headers = OrderedDict(("X-Ops-Authorization-{0}".format(i+1), chunk) for i, chunk in enumerate(chunks(signed_request, 60)))
    
    all_headers = OrderedDict(headers)
    all_headers.update(auth_headers)
    
    # print('curl '+' \\\n'.join("-H {0}: {1}".format(i[0], i[1]) for i in all_headers.items())+" \\\nhttps://chef.local/nodes")
    
    print(requests.get("https://chef.local"+path, headers=all_headers).text)
    

At each step I tried to check if the variables have the same result as their
counterparts in the curl script.

The problem seems to be at signing stage - there's an obvious discrepancy
between the output of python's packages and my mac's openssl tools. Due to
this discrepancy, Chef returns `{"error":["Invalid signature for user or
client 'anton'"]}`. Curl script with the same values and keys works fine.

`dummy_sign = base64.b64encode(rsa.sign("hello".encode(), privkey, "SHA-1"))`
from Python has the value of

    
    
    N7QSZRD495vV9cC35vQsDyxfOvbMN3TcnU78in911R54IwhzPUKnJTdFZ4D/KpzyTVmVBPoR4nY5um9QVcihhqTJQKy+oPF+8w61HyR7YyXZRqmx6sjiJRffC4uOGb5Wjot8csAuRSeUuHaNTl6HCcfRKnwUZnB7SctKoK6fXv0skWN2CzV9CjfHByct3oiy/xAdTz6IB+fLIwSQUf1k7lJ4/CmLJLP/Gu/qALkvWOYDAKxmavv3vYX/kNhzApKgTYPMw6l5k1aDJGRVm9Ch/BNQbg1WfZiT6LK+m4KAMFbTORfEH45KGWBCj9zsyETyMCAtUycebjqMujMqEwzv7w==
    

while the output of `echo -n "hello" | openssl rsautl -sign -inkey ~/.chef/anton.pem | openssl enc -base64` is 
    
    
    WfoASF1f5DPT3CVPlWDrIiTwuEnjr5yCV+WIlbQLFmwm3nfhIqfTPLyTM56SwTSg
    CKdboVU4EBFxC3RsU2aPpELqRH6+Fnl2Tl273vo6kLzvC/8+tUBTdNZdzSPhx6S8
    x+6wzVFXsd3QeGAWoHkEgTKodSByFzARnZFxO2JzUe4dnygijwruHdf9S4ldrRo6
    eaShwaxuNzM0cIl+Umz5iym3cCD6GFL13njmXZs3cHRLesBtLKA7pNxJ1UDf2WN2
    OK09aK+bHaM4jl5HeQ2SdNzBQIKvyDcxX4Divnf2I/0tzD16J6BEMGCfTfsI2f3K
    TVGulq81+sH9zo8lGnpDrw==
    

I couldn't find the information on default hashing algorithm in openssl for
`rsautl`, but I guess it's SHA-1.

At this point I don't really know which way to look, hope anyone can help make
it right.


Answer: From [Chef Authentication and Authorization with
cURL](https://docs.chef.io/auth.html#curl),

    
    
    timestamp=$(date -u "+%Y-%m-%dT%H:%M:%SZ")  
    

time is in UTC, so in Python, it has to be

    
    
    timestamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")  
    

`openssl` equivalent of Python,

    
    
    dummy_sign = base64.b64encode(rsa.sign("hello".encode(), privkey, "SHA-1"))  
    

is

    
    
    echo -n hello|openssl dgst -sha1 -sign ~/.chef/anton.pem -keyform PEM|openssl enc -base64
    

In Python code, you're signing the message digest, SHA-1, of the message.
That's called detached signature.

`echo -n "hello" | openssl rsautl -sign -inkey ~/.chef/anton.pem | openssl enc -base64` but this one signs the whole message, without making digest. 

Python `rsa` module has no equivalent of `openssl rsautl -sign`. So I defined
a function to fill that space.

    
    
    from rsa import common, transform, core, varblock
    from rsa.pkcs1 import _pad_for_signing
    
    def pure_sign(message, priv_key):
        '''Signs the message with the private key.
    
        :param message: the message to sign. Can be an 8-bit string or a file-like
            object. If ``message`` has a ``read()`` method, it is assumed to be a
            file-like object.
        :param priv_key: the :py:class:`rsa.PrivateKey` to sign with
        :return: a message signature block.
        :raise OverflowError: if the private key is too small to contain the
            requested hash.
    
        '''
    
        keylength = common.byte_size(priv_key.n)
        padded = _pad_for_signing(message, keylength)
    
        payload = transform.bytes2int(padded)
        encrypted = core.encrypt_int(payload, priv_key.d, priv_key.n)
        block = transform.int2bytes(encrypted, keylength)
    
        return block
    

Test;  
openssl

    
    
    echo -n hello|openssl rsautl -sign -inkey .chef/anton.pem |base64  
    
    foIy6HVpfIpNk4hMYg8YWCEZwZ7w4Qexr6KXDbJ7/vr5Jym56joofkn1qUak57iSercqQ1xqBsIT
    fo6bDs2suYUKu15nj3FRQ54+LcVKjDrUUEyl2kfJgVtXLsdhzYj1SBFJZnbz32irVMVytARWQusy
    b2f2GQKLTogGhCywFFyhw5YpAHmKc2CQIHw+SsVngcPrmVAAtvCZQRNV5zR61ICipckNEXnya8/J
    Ga34ntyELxWDradY74726OlJSgszpHbAOMK02C4yx7OU32GWlPlsZBUGAqS5Tu4MSjlD1f/eQBsF
    x/pn8deP4yuR1294DTP7dsZ9ml64ZlcIlg==
    

Python

    
    
    base64.b64encode(pure_sign.pure_sign(b'hello',prik)).decode()
    
    'foIy6HVpfIpNk4hMYg8YWCEZwZ7w4Qexr6KXDbJ7/vr5Jym56joofkn1qUak57iSercqQ1xqBsITfo6bDs2suYUKu15nj3FRQ54+LcVKjDrUUEyl2kfJgVtXLsdhzYj1SBFJZnbz32irVMVytARWQusyb2f2GQKLTogGhCywFFyhw5YpAHmKc2CQIHw+SsVngcPrmVAAtvCZQRNV5zR61ICipckNEXnya8/JGa34ntyELxWDradY74726OlJSgszpHbAOMK02C4yx7OU32GWlPlsZBUGAqS5Tu4MSjlD1f/eQBsFx/pn8deP4yuR1294DTP7dsZ9ml64ZlcIlg=='
    

Change the line;

    
    
    signed_request = base64.b64encode(rsa.sign(canonical_request.encode(), privkey, "SHA-1"))
    

to

    
    
    signed_request = base64.b64encode(pure_sign(canonical_request.encode(), privkey))
    

