~mcepl/m2crypto#9: 
subjectKeyIdentifier and authorityKeyIdentifier segfault

Migrated from: https://gitlab.com/m2crypto/m2crypto/-/issues/9
Created by: Matěj Cepl mcepl@cepl.eu
Created at: 2015-07-08T21:40:02.514Z
Milestone: 0.42

NDavidBrown: The bug is described here.

It would be good if we could specify the typical values

'authorityKeyIdentifier': 'keyid, issuer', 'subjectKeyIdentifier' : 'hash',

..and they actually work.

Someone needs to integrate the bugfix described here.

Status
REPORTED
Submitter
~mcepl
Assigned to
No-one
Submitted
11 months ago
Updated
14 hours ago
Labels
milestone-0.46

~mcepl 11 months ago

On 2015-10-16T19:07:47.809Z, Matěj Cepl wrote:

Log from Travis-CI run

log.txt

~mcepl 11 months ago*

On 2015-10-16T20:22:22.797Z, Matěj Cepl wrote:

Log when running the test in valgrind -v valgrind-log-python-crash.txt

~mcepl 11 months ago

Changed on 2016-03-20T22:06:43.832Z by Matěj Cepl:

Milestone changed to 0.25.0

~mcepl 11 months ago

Changed on 2016-07-12T19:16:07.428Z by Matěj Cepl:

Milestone changed to 0.28

~mcepl 11 months ago

On 2016-08-16T20:26:18.412Z, Jesse Peterson wrote:

WIth the common workaround for this issue the SWIG function x509v3_lhash and x509v3_set_conf_lhash are used. However those appear to have been removed. Without having the issue fixed, and taking away the workaround capability, it seems this bug is now a show-stopper for anything that wants to sign certificates and have an AKI extension, I think. Meaning no CAs with M2Crypto.

~mcepl 11 months ago

On 2016-08-21T19:39:55.906Z, Matěj Cepl wrote:

LHASH-stuff is a horrible mess. Removal of it was the best thing I did, IMHO.

You can try to use it, but ALL tests which are currently passing must still pass (including on RHEL-6, Windows, and python3). I am not willing to touch that stuff evermore if I can avoid it.

~mcepl 11 months ago

Changed on 2017-10-05T23:29:44.335Z by Matěj Cepl:

changed milestone to 0.29

~mcepl 11 months ago

On 2017-10-05T23:29:57.193Z, Matěj Cepl wrote:

@jessepeterson I am sorry for letting this bug rot for so long, but could you please explain me what’s the problem. Unfortunately, OSAF bugzilla went the way of Dodo, and I don’t see anything I could analyse here.

~mcepl 11 months ago

Changed on 2017-10-05T23:29:57.318Z by Matěj Cepl:

added needinfo label

~mcepl 11 months ago

Changed on 2017-10-13T03:06:58.895Z by Matěj Cepl:

mentioned in issue #83

~mcepl 11 months ago

On 2017-10-17T02:52:30.757Z, Jesse Peterson wrote:

Hello @mcepl! I duplicated the workaround found in that OSAF bugzilla report here:

https://github.com/jessepeterson/commandment/blob/master/commandment/pki/m2fix.py

Basically it was a problem with initializing some structures that we take care of manually using ctypes. Thanks!

(Last edited at 2020-02-02T18:25:57.503Z.)

~mcepl 11 months ago

On 2017-10-17T13:22:06.473Z, Matěj Cepl wrote:

So, effectively what you ask for is that the structure ctx would be initialized in x509v3_set_nconf with NULLs for issuer_cert, subject_cert, subject_req, and crl (issuer_cert can be later filled with set_issuer_name method of X509 object), right?

~mcepl 11 months ago

On 2017-10-18T03:26:16.303Z, Jesse Peterson wrote:

It's been a long while since I've looked at it, myself. But without the Python fix the operations would cause a segfault when trying to set the SKI/AKI details, I believe.

~mcepl 11 months ago

Changed on 2018-02-23T21:19:45.132Z by Matěj Cepl:

removed milestone

~mcepl 11 months ago

On 2020-02-02T18:25:57.412Z, Matěj Cepl wrote:

@jessepeterson Could I ask for attaching that file here to this issue ticket, please? Unfortunately, everything else seems to change too fast to my tortoise speed.

~mcepl 11 months ago*

On 2020-02-12T10:09:47.132Z, Matěj Cepl wrote:

OK, somebody else provided me with this example crash_m2crypto.py.

~@stitny$ python3 crash_m2crypto.py 
This next bit seq faults.
~@stitny$ python crash_m2crypto.py 
This next bit seq faults.
Traceback (most recent call last):
  File "crash_m2crypto.py", line 49, in <module>
    M2CRYPTO_X509.new_extension('authorityKeyIdentifier', 'keyid')
  File "/usr/lib64/python2.7/site-packages/M2Crypto/X509.py", line 47, in new_extension
    x509_ext_ptr = m2.x509v3_ext_conf(None, ctx, name, value)
M2Crypto.X509.X509Error: no issuer certificate
~@stitny$

Interesting, so it crashes for me with python2.7, but not python3.7. And I don’t get crashing Python interpreter either.

(Last edited at 2020-02-12T11:23:38.635Z.)

~mcepl 11 months ago

On 2020-02-12T17:34:52.112Z, Scott A Brown wrote:

'Twas I. Some more details; not sure how to determine the version of SSL libraries linked in?

On CentOS 7:

(m2crypto) $ python m2crash.py
This next bit seq faults.
Segmentation fault
(m2crypto) $ pip freeze
M2Crypto==0.35.2
(m2crypto) $ python --version
Python 3.7.4


$ ldconfig -p | grep ssl
libssl3.so (libc6,x86-64) => /lib64/libssl3.so
libssl.so.10 (libc6,x86-64) => /lib64/libssl.so.10
libssl.so.10 (libc6) => /lib/libssl.so.10
libssl.so (libc6,x86-64) => /lib64/libssl.so
libevent_openssl-2.0.so.5 (libc6,x86-64) => /lib64/libevent_openssl-2.0.so.5
libevent_openssl-2.0.so.5 (libc6) => /lib/libevent_openssl-2.0.so.5

On MacOS 10.13.6:

(m2crypto) $ python m2crash.py
This next bit seq faults.
Segmentation fault: 11
(m2crypto) $ pip freeze
M2Crypto==0.35.2
(m2crypto) $ python --version
Python 3.6.5

$ ls /usr/lib/*ssl*
/usr/lib/libboringssl.dylib	/usr/lib/libssl.0.9.8.dylib	/usr/lib/libssl.43.dylib
/usr/lib/libssl.0.9.7.dylib	/usr/lib/libssl.35.dylib	/usr/lib/libssl.dylib

(Last edited at 2020-02-12T17:36:57.000Z.)

~mcepl referenced this from #9 11 months ago

~mcepl 11 months ago

On 2020-02-12T17:56:22.998Z, Matěj Cepl wrote:

  python36-m2crypto.x86_64 0:0.35.2-5.el7                                                    
~@narcis$ gdb python3.6 crash_m2crypto.py 

Program received signal SIGSEGV, Segmentation fault.
0x00007fffef6b2143 in X509_get_ext_by_NID (x=x@entry=0x7ffff7d6a5a0 <PyUnicode_Type>, 
    nid=nid@entry=82, lastpos=lastpos@entry=-1) at x509_ext.c:121
121	    return (X509v3_get_ext_by_NID(x->cert_info->extensions, nid, lastpos));
(gdb) t a a bt

Thread 1 (Thread 0x7ffff7fd9740 (LWP 11896)):
#0  0x00007fffef6b2143 in X509_get_ext_by_NID (x=x@entry=0x7ffff7d6a5a0 <PyUnicode_Type>, 
    nid=nid@entry=82, lastpos=lastpos@entry=-1) at x509_ext.c:121
<a href="/~mcepl/m2crypto/1" title="~mcepl/m2crypto#1: Delete this Repo">#1</a>  0x00007fffef6bccde in v2i_AUTHORITY_KEYID (method=<optimized out>, ctx=0x7fffeb7d68b8, 
    values=<optimized out>) at v3_akey.c:160
<a href="/~mcepl/m2crypto/2" title="~mcepl/m2crypto#2: added PEM_write_bio_SSL_SESSION">#2</a>  0x00007fffef6b7a29 in do_ext_nconf (conf=conf@entry=0x7fffffffd7d0, 
    ctx=ctx@entry=0x7fffeb7d68b8, ext_nid=90, crit=crit@entry=0, 
    value=value@entry=0x876b70 "keyid") at v3_conf.c:147
<a href="/~mcepl/m2crypto/3" title="~mcepl/m2crypto#3: setup.py: Fix custom install class to not force an egg install">#3</a>  0x00007fffef6b7ca6 in X509V3_EXT_nconf (conf=conf@entry=0x7fffffffd7d0, 
    ctx=ctx@entry=0x7fffeb7d68b8, name=name@entry=0x7d1ed0 "authorityKeyIdentifier", 
    value=value@entry=0x876b70 "keyid") at v3_conf.c:93
<a href="/~mcepl/m2crypto/4" title="~mcepl/m2crypto#4: X509 subject_hash missing">#4</a>  0x00007fffef6b813f in X509V3_EXT_conf (conf=<optimized out>, ctx=0x7fffeb7d68b8, 
    name=0x7d1ed0 "authorityKeyIdentifier", value=0x876b70 "keyid") at v3_conf.c:472
<a href="/~mcepl/m2crypto/5" title="~mcepl/m2crypto#5: httpslib.HTTPSConnection is broken on py3">#5</a>  0x00007fffefc6b79e in x509v3_ext_conf ()
   from /usr/lib64/python3.6/site-packages/M2Crypto/_m2crypto.cpython-36m-x86_64-linux-gnu.so
<a href="/~mcepl/m2crypto/6" title="~mcepl/m2crypto#6: First run of setup.py fails when setting up M2Crypto">#6</a>  0x00007fffefc6b93a in _wrap_x509v3_ext_conf ()
   from /usr/lib64/python3.6/site-packages/M2Crypto/_m2crypto.cpython-36m-x86_64-linux-gnu.so
<a href="/~mcepl/m2crypto/7" title="~mcepl/m2crypto#7: Crash when verify_fail_if_no_peer_cert and client does not provide cert">#7</a>  0x00007ffff799904a in _PyCFunction_FastCallDict () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/8" title="~mcepl/m2crypto#8: Restack">#8</a>  0x00007ffff7a04a3f in call_function () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/9" title="~mcepl/m2crypto#9: subjectKeyIdentifier and authorityKeyIdentifier segfault">#9</a>  0x00007ffff79f90a7 in _PyEval_EvalFrameDefault () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/10" title="~mcepl/m2crypto#10: Enable AES CTR mode">#10</a> 0x00007ffff7a03a7c in _PyEval_EvalCodeWithName () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/11" title="~mcepl/m2crypto#11: Any schedule to merge https://github.com/M2Crypto/M2Crypto/ and https://github.com/martinpaljak/M2Crypto?">#11</a> 0x00007ffff7a048da in fast_function () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/12" title="~mcepl/m2crypto#12: Unable to run pyenv">#12</a> 0x00007ffff7a04b63 in call_function () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/13" title="~mcepl/m2crypto#13: Hostname Matching interface needed">#13</a> 0x00007ffff79f90a7 in _PyEval_EvalFrameDefault () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/14" title="~mcepl/m2crypto#14: AltName interface required">#14</a> 0x00007ffff7a04e62 in PyEval_EvalCodeEx () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/15" title="~mcepl/m2crypto#15: Use swig generated python loader">#15</a> 0x00007ffff7a05afb in PyEval_EvalCode () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/16" title="~mcepl/m2crypto#16: Building problems under PyEnv, Yosemite">#16</a> 0x00007ffff7a8f1ee in run_mod () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/17" title="~mcepl/m2crypto#17: Where does 0.22.3 live?">#17</a> 0x00007ffff793a464 in PyRun_FileExFlags () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/18" title="~mcepl/m2crypto#18: Document X509 version is 0-index based">#18</a> 0x00007ffff793a836 in PyRun_SimpleFileExFlags () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/19" title="~mcepl/m2crypto#19: AttributeError: &#39;module&#39; object has no attribute &#39;PKCS5_SALT_LEN&#39;">#19</a> 0x00007ffff7a956a2 in Py_Main () from /lib64/libpython3.6m.so.1.0
<a href="/~mcepl/m2crypto/20" title="~mcepl/m2crypto#20: cannot load certificate from string">#20</a> 0x0000000000400a99 in main ()
(gdb)

~mcepl 11 months ago

On 2020-02-17T20:27:18.614Z, Matěj Cepl wrote:

Hmm, the only thing which comes to my mind is that the crash is with OpenSSL 1.0.2k, where it doesn't crash with 1.1.1d.

~mcepl 11 months ago

On 2021-08-04T00:18:18.077Z, Scott A Brown wrote:

This seems to refer to a workaround for this issue: https://github.com/CyberShadow/commandment/blob/master/commandment/pki/m2fix.py

~mcepl 11 months ago

Changed on 2023-11-01T22:27:52.566Z by Matěj Cepl:

removed time estimate

(Last edited at 2023-11-01T22:27:52.572Z.)

~mcepl referenced this from #73 11 months ago

~mcepl referenced this from #83 11 months ago

~mcepl added milestone-0.42 11 months ago

~mcepl 11 months ago

m2fix.py is:

'''
Copyright (c) 2016 Jesse Peterson
Licensed under the MIT license. See the included LICENSE.txt file for details.
'''

from M2Crypto import m2
from M2Crypto.X509 import X509_Extension
from ctypes import *

class X509V3_CTX(Structure):
    _fields_ = [
        ('flags', c_int),
        ('issuer_cert', c_void_p),
        ('subject_cert', c_void_p),
        ('subject_req', c_void_p),
        ('crl', c_void_p),
        ('db_meth', c_void_p),
        ('db', c_void_p),
    ]

def fix_ctx(m2_ctx, issuer=None):
    '''Fix for setting authorityKeyIdentifier with issuer certificate.

    Workaround sourced from:
      https://bugzilla.osafoundation.org/show_bug.cgi?id=7530#c13

    Thanks to Matt in comment <a href="/~mcepl/m2crypto/13" title="~mcepl/m2crypto#13: Hostname Matching interface needed">#13</a> for the workaround of setting the context
    issuer and zeroing the structure. This fixes a segfault that would
    otherwise happen when setting the extension.'''
    ctx = X509V3_CTX.from_address(long(m2_ctx))

    ctx.flags = 0
    ctx.issuer_cert = long(issuer.x509) if issuer else None
    ctx.subject_cert = None
    ctx.subject_req = None
    ctx.crl = None

def new_extension_fixed(name, value, critical=0, issuer=None, _pyfree=1):
    """
    Create new X509_Extension instance with fix for issuer setting.
    """
    if name == 'subjectKeyIdentifier' and \
        value.strip('0123456789abcdefABCDEF:') is not '':
        raise ValueError('value must be precomputed hash')
    lhash = m2.x509v3_lhash()
    ctx = m2.x509v3_set_conf_lhash(lhash)

    # zero out structure, assign issuer
    fix_ctx(ctx, issuer)

    x509_ext_ptr = m2.x509v3_ext_conf(lhash, ctx, name, value)

    if x509_ext_ptr is None:
        raise Exception

    x509_ext = X509_Extension(x509_ext_ptr, _pyfree)
    x509_ext.set_critical(critical)
    return x509_ext

~mcepl 9 months ago

Crash is in X509_get_ext_by_NID, isn’t this significant?

~mcepl removed milestone-0.42 9 months ago

~mcepl added milestone-0.43 9 months ago

~mcepl referenced this from #9 8 months ago

~mcepl 8 months ago 

Program received signal SIGSEGV, Segmentation fault.
X509_get0_pubkey (x=0x1a) at crypto/x509/x509_cmp.c:385
385	crypto/x509/x509_cmp.c: Adresář nebo soubor neexistuje.
(gdb) t a a bt

Thread 1 (Thread 0x7ffff7f9cb80 (LWP 18115) "python3"):
#0  X509_get0_pubkey (x=0x1a) at crypto/x509/x509_cmp.c:385
<a href="/~mcepl/m2crypto/1" title="~mcepl/m2crypto#1: Delete this Repo">#1</a>  0x00007ffff6fbd0d9 in X509_check_private_key (x=<optimized out>, k=0xa2e) at crypto/x509/x509_cmp.c:400
<a href="/~mcepl/m2crypto/2" title="~mcepl/m2crypto#2: added PEM_write_bio_SSL_SESSION">#2</a>  0x00007ffff6fa95f3 in v2i_AUTHORITY_KEYID (method=<optimized out>, ctx=0x7ffff76d63f0, values=<optimized out>) at crypto/x509/v3_akid.c:151
<a href="/~mcepl/m2crypto/3" title="~mcepl/m2crypto#3: setup.py: Fix custom install class to not force an egg install">#3</a>  0x00007ffff6fac61d in do_ext_nconf (conf=0x5555557487b0, ctx=0x7ffff76d63f0, ext_nid=90, crit=0, value=0x555555748120 "keyid") at crypto/x509/v3_conf.c:108
<a href="/~mcepl/m2crypto/4" title="~mcepl/m2crypto#4: X509 subject_hash missing">#4</a>  0x00007ffff6fac8cb in X509V3_EXT_nconf_int (conf=0x5555557487b0, ctx=0x7ffff76d63f0, section=0x0, name=0x555555748ae0 "authorityKeyIdentifier", value=<optimized out>) at crypto/x509/v3_conf.c:45
<a href="/~mcepl/m2crypto/5" title="~mcepl/m2crypto#5: httpslib.HTTPSConnection is broken on py3">#5</a>  0x00007ffff6fac9bd in X509V3_EXT_nconf (conf=0x5555557487b0, ctx=0x7ffff76d63f0, name=0x555555748ae0 "authorityKeyIdentifier", value=0x555555748120 "keyid") at crypto/x509/v3_conf.c:61
<a href="/~mcepl/m2crypto/6" title="~mcepl/m2crypto#6: First run of setup.py fails when setting up M2Crypto">#6</a>  0x00007ffff6fad03f in X509V3_EXT_conf (conf=0x0, ctx=0x7ffff76d63f0, name=0x555555748ae0 "authorityKeyIdentifier", value=0x555555748120 "keyid") at crypto/x509/v3_conf.c:489
<a href="/~mcepl/m2crypto/7" title="~mcepl/m2crypto#7: Crash when verify_fail_if_no_peer_cert and client does not provide cert">#7</a>  0x00007ffff73a49e5 in x509v3_ext_conf (value=<optimized out>, name=0x555555748ae0 "authorityKeyIdentifier", ctx=0x7ffff76d63f0, conf=<optimized out>) at src/SWIG/_m2crypto_wrap.c:9083
<a href="/~mcepl/m2crypto/8" title="~mcepl/m2crypto#8: Restack">#8</a>  _wrap_x509v3_ext_conf (self=<optimized out>, args=<optimized out>) at src/SWIG/_m2crypto_wrap.c:27193
<a href="/~mcepl/m2crypto/9" title="~mcepl/m2crypto#9: subjectKeyIdentifier and authorityKeyIdentifier segfault">#9</a>  0x00007ffff7cb903d in _PyCFunction_FastCallDict (func_obj=<built-in method x509v3_ext_conf of module object at remote 0x7ffff76a1a48>, args=0x5555556d5000, nargs=<optimized out>, kwargs=<optimized out>) at Objects/methodobject.c:234
<a href="/~mcepl/m2crypto/10" title="~mcepl/m2crypto#10: Enable AES CTR mode">#10</a> 0x00007ffff7d2aa22 in call_function (pp_stack=pp_stack@entry=0x7fffffffd998, oparg=<optimized out>, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
<a href="/~mcepl/m2crypto/11" title="~mcepl/m2crypto#11: Any schedule to merge https://github.com/M2Crypto/M2Crypto/ and https://github.com/martinpaljak/M2Crypto?">#11</a> 0x00007ffff7d2ee85 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
<a href="/~mcepl/m2crypto/12" title="~mcepl/m2crypto#12: Unable to run pyenv">#12</a> 0x00007ffff7d2a486 in _PyEval_EvalCodeWithName (_co=<code at remote 0x7ffff769ec00>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x5555555b6ba8, kwcount=0, kwstep=1, defs=0x7ffff7254120, defcount=2, kwdefs=0x0, closure=0x0, name='new_extension', qualname='new_extension') at Python/ceval.c:4166
<a href="/~mcepl/m2crypto/13" title="~mcepl/m2crypto#13: Hostname Matching interface needed">#13</a> 0x00007ffff7d2a6ff in fast_function (func=<optimized out>, stack=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:4992
<a href="/~mcepl/m2crypto/14" title="~mcepl/m2crypto#14: AltName interface required">#14</a> 0x00007ffff7d2a905 in call_function (pp_stack=pp_stack@entry=0x7fffffffdc78, oparg=<optimized out>, kwnames=kwnames@entry=0x0) at Python/ceval.c:4872
<a href="/~mcepl/m2crypto/15" title="~mcepl/m2crypto#15: Use swig generated python loader">#15</a> 0x00007ffff7d2ee85 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
<a href="/~mcepl/m2crypto/16" title="~mcepl/m2crypto#16: Building problems under PyEnv, Yosemite">#16</a> 0x00007ffff7d2a486 in _PyEval_EvalCodeWithName (_co=_co@entry=<code at remote 0x7ffff7a8b930>, globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), locals=locals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0--Type <RET> for more, q to quit, c to continue without paging-- 
:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4166
<a href="/~mcepl/m2crypto/17" title="~mcepl/m2crypto#17: Where does 0.22.3 live?">#17</a> 0x00007ffff7d2a5c3 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=locals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), _co=_co@entry=<code at remote 0x7ffff7a8b930>) at Python/ceval.c:4187
<a href="/~mcepl/m2crypto/18" title="~mcepl/m2crypto#18: Document X509 version is 0-index based">#18</a> PyEval_EvalCode (co=co@entry=<code at remote 0x7ffff7a8b930>, globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), locals=locals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKC--Type <RET> for more, q to quit, c to continue without paging--
AgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated)) at Python/ceval.c:731
<a href="/~mcepl/m2crypto/19" title="~mcepl/m2crypto#19: AttributeError: &#39;module&#39; object has no attribute &#39;PKCS5_SALT_LEN&#39;">#19</a> 0x00007ffff7d53ef0 in run_mod (mod=mod@entry=0x555555619d98, filename=filename@entry='crash_m2crypto.py', globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), locals=locals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), flags=flags@entry=0x7fffffffdf7c, arena=arena@entry=0x7ffff7ae72b8) at Python/pythonrun.c:1025
<a href="/~mcepl/m2crypto/20" title="~mcepl/m2crypto#20: cannot load certificate from string">#20</a> 0x00007ffff7d55d21 in PyRun_FileExFlags (fp=0x555555558320, filename_str=<optimized out>, start=<optimized out>, globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='crash_m2crypto.py') at remote 0x7ffff7a89ac8>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7f3d688>, '__file__': 'crash_m2crypto.py', '__cached__': None, 'M2CRYPTO_X509': <module at remote 0x7ffff76ec778>, 'sub_key_id': '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64', 'cert_pem_string': '\n-----BEGIN CERTIFICATE-----\nMIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV\nBAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK\nDApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl\nczERMA8GA1UEAwwIbG9jYWxfY2EwHhcNMTgwNTMxMTgwOTMwWhcNMzcxMjMxMTgw\nOTMwWjB7MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExEjAQBgNVBAcMCUxvcyBH\nYXRvczETMBEGA1UECgwKTmFKZURhIExMQzEjMCEGA1UECwwaTmFKZURhIEdlb2Nh\nY2hpbmcgU2VydmljZXMxETAPBgNVBAMMCGxvY2FsX2NhMIICIjANBgkqhkiG9w0B\nAQEFAAOCAg8AMIICCgKCAgEAwL4VBqghrv9DdUq+63Yty/kaNINIO+ldhY8GxrZd\nKXdJqa...(truncated), closeit=1, flags=0x7fffffffdf7c) at Python/pythonrun.c:978
<a href="/~mcepl/m2crypto/21" title="~mcepl/m2crypto#21: tests for sha256 / sha512 X509 cert signatures missing?">#21</a> 0x00007ffff7d55e7d in PyRun_SimpleFileExFlags (fp=0x555555558320, filename=<optimized out>, closeit=1, flags=0x7fffffffdf7c) at Python/pythonrun.c:419
<a href="/~mcepl/m2crypto/22" title="~mcepl/m2crypto#22: Should we do something around POODLE?">#22</a> 0x00007ffff7d6cfe1 in run_file (p_cf=0x7fffffffdf7c, filename=0x55555555c0d0 L"crash_m2crypto.py", fp=0x555555558320) at Modules/main.c:351
<a href="/~mcepl/m2crypto/23" title="~mcepl/m2crypto#23: Fixing memory leaks in the SWIG wrapper.">#23</a> Py_Main (argc=argc@entry=2, argv=argv@entry=0x5555555582a0) at Modules/main.c:821
--Type <RET> for more, q to quit, c to continue without paging--
<a href="/~mcepl/m2crypto/24" title="~mcepl/m2crypto#24: Support for TLSv1.1/TLSv1.2">#24</a> 0x0000555555554c90 in main (argc=2, argv=<optimized out>) at ./Programs/python.c:102
(gdb)

~mcepl 8 months ago*

Start of the work is in the branch https://git.sr.ht/~mcepl/m2crypto/log/9_X509-new_extension

However, this solution doesn’t incorporate the switch from m2.x509v3_set_conf_lhash to m2.x509v3_set_nconf in 332c580fb512.

Also, crash_m2crypto.py should be reworked into a test case.

~mcepl 7 months ago

Matěj Cepl referenced this ticket in commit c52f588.

~mcepl 7 months ago

Matěj Cepl referenced this ticket in commit ee381da.

~mcepl 7 months ago

Matěj Cepl referenced this ticket in commit 9e28d84.

~mcepl 7 months ago

Matěj Cepl referenced this ticket in commit bd1e730.

~mcepl 7 months ago

Matěj Cepl referenced this ticket in commit 423c8cb.

~mcepl removed milestone-0.43 6 months ago

~mcepl added milestone-0.44 6 months ago

~mcepl 5 months ago

Matěj Cepl referenced this ticket in commit 39256e8.

~mcepl 5 months ago

Matěj Cepl referenced this ticket in commit f4f1504.

~mcepl removed milestone-0.44 3 months ago

~mcepl added milestone-0.45 3 months ago

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 74465d6.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 3e33943.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 25d3b28.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit bc74131.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 3eabe81.

~mcepl REPORTED FIXED a month ago

Matěj Cepl referenced this ticket in commit f516ea6.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 6bafc31.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 13fc599.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit 6fe4fe6.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit a5f51b8.

~mcepl a month ago

Matěj Cepl referenced this ticket in commit b0249f3.

~mcepl FIXED FIXED a month ago

Matěj Cepl referenced this ticket in commit 847cde2.

~mcepl FIXED REPORTED a month ago

~mcepl a month ago

Still, not done.

~mcepl added milestone-0.46 a month ago

~mcepl removed milestone-0.45 a month ago

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit 97abda3.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit 648f2b2.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit 8797bfe.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit d459c6e.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit ab2b6dc.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit 3cb4bc9.

~mcepl 12 days ago

Matěj Cepl referenced this ticket in commit 6f6c1ab.

~mcepl 14 hours ago

The provided GDB backtrace indicates a segmentation fault originating within OpenSSL's X509_get0_pubkey function, which is called during the processing of an authorityKeyIdentifier extension. This suggests an issue with how M2Crypto is interacting with OpenSSL when creating or handling X509 extensions, specifically authorityKeyIdentifier and potentially subjectKeyIdentifier.

Here's a breakdown of the problem and potential reasons:

  • X509_get0_pubkey and X509_check_private_key: These are core OpenSSL functions related to public key operations within X509 certificates. The fact that the crash occurs here, with an x argument of 0x1a (which is a very small, likely invalid, memory address), points to a corrupted or invalid X509 object being passed to these functions.
  • v2i_AUTHORITY_KEYID: This OpenSSL function is responsible for converting the string representation of an Authority Key Identifier into its internal X509 structure. The crash happening within this function, or immediately after it, suggests the input keyid (which you've provided as the string 'keyid') might be causing issues during parsing or when it attempts to look up or associate with a public key.
  • M2CRYPTO_X509.new_extension('authorityKeyIdentifier', 'keyid'): This is the line that directly precedes the crash. The value you are passing, 'keyid', is likely not in the expected format for an authorityKeyIdentifier extension. Authority Key Identifier extensions typically refer to the subject key identifier of the issuing certificate or a hash of the issuer's public key. A simple string like 'keyid' is not a valid representation.

Why M2CRYPTO_X509.new_extension('subjectKeyIdentifier', sub_key_id) doesn't crash:

The subjectKeyIdentifier extension typically takes a hex-encoded string of the public key hash. Your sub_key_id variable is in this format: '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64'. M2Crypto/OpenSSL is likely able to parse this correctly.

The Solution:

The problem is that you are passing an invalid string for the authorityKeyIdentifier extension. The string 'keyid' is not a valid format for this extension.

To correctly create an authorityKeyIdentifier extension, you need to provide a value that OpenSSL can interpret. Common forms for authorityKeyIdentifier include:

  1. keyid followed by the Subject Key Identifier of the issuer: This is the most common and usually preferred method, where you specify keyid: <hex_string_of_issuer_ski>.
  2. keyid:always: This tells OpenSSL to derive the Authority Key Identifier from the issuer's public key automatically. However, this method might require the issuer's certificate to be available in the context.
  3. issuer or issuer:always: This refers to the issuer's certificate and its serial number.
  4. keyid,issuer:always: A combination.

Since you've loaded a local_ca certificate, which seems to be self-signed (or at least the issuer and subject are the same), you can use its subjectKeyIdentifier for the authorityKeyIdentifier extension.

Here's how to fix your script:

from M2Crypto import X509 as M2CRYPTO_X509

sub_key_id = '1C:E6:F0:58:58:32:BC:7B:BA:8E:E0:23:1B:FF:17:99:B0:4D:CF:64'

cert_pem_string = """
-----BEGIN CERTIFICATE-----
MIIGFjCCA/6gAwIBAgIJAO7rHaO9YDQDMA0GCSqGSIb3DQEBCwUAMHsxCzAJBgNV
BAYTAlVTMQswCQYDVQQIDAJDQTESMBAGA1UEBwwJTG9zIEdhdG9zMRMwEQYDVQQK
DApOYUplRGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNl
czERMA8GAgNVBAMMCGxvY2FsX2NhMB4XDTE4MDUzMTE4MDkzMFoXDTM3MTIzMTE4
MDkzMFowezELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRMwEQYDVQQKDAppYWpl
RGEgTExDMSMwIQYDVQQLDBpOYUplRGEgR2VvY2FjaGluZyBTZXJ2aWNlczERMA8G
A1UEAwwIbG9jYWxfY2EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDA
vhUGqCGu/0N1Sr7rdi3L+Ro0g0g76V2FjwbGtl0pd0mpqdkt0nNZaW4lys7OTGKm
l2TzL1JiOueChky5H+8vbmXF8Mp2j3DIQRlYQaem/cijW4Q8QRiUNdsIcB6pB7O
a7JvyTxMsbwQC2MlE9ItNR2zJ1RMpzGvRoO+wheZ8zPtXquo+/rJfzoxd2G6/L9R
rwo3Izgwb6NiXbQadg675o/0shmhD1LJT5DcmvjL0shj+VasUKAOwgt/5GtkjjeE
53VsExOkJKrH/RUodl6dXiXBq4ehtwdFGQnTDpygbKSXI8M3FPU/zYjt4HYDW5R+
VlkKYEdMaOwQ4b9waArCZPSh6SoSZ8SyjRN1wKqnda+vl5MrzPGTbnN8CXvs89+t
i+iT+pdsCc/L3kdwqaV3HNE0pjTg3bChWJ/iNltFE3lqTmnUcMfvhHpAcj6txB4Y
qzvQgh7DQ4KnKwFgXvrS7t2fdgFxVe/Bl7ApXi1tEg8AEjasuLqb/sTWLyvoWog1
iJg7uWsv7F3DXloc7q80eAh610KtPTSzcCvOfd9Ii6+P9yxcOZv4vCbX7rrCJt/s
cL2/Hz1qqQYA/DcKtvitA2hUAx6AJlMvW14Dw3tcnHny5lJ4Ty7ZiGN7Bg9Jj1uJ
usa0le2dkFwx5WjXZu2QMtjgJT8aBl3iM36jNtY7v6cCAwEAATANBgkqhkiG9w0B
AQEFAAOCAgEAepM/VwzIN3aWc08IgF0+J3wYAjDzq2y/ixDXwL/B/XOHElySaiDa
kiT6HM52Ek/LkFK67LlpTZIxCwViBxkkcTBS10ymGfsYY5R7lOx14SUIXPOS/Pvh
t1IZBuSp5J9woZjEZitkInmWYSmA2Q85JtFs86pNQD9gCOCd5hnKK2LqOwrPAcnO
J06FhZFT/psI5MR8XFjD/dJUfnkxbK6S77sCslALdsaYdWp6B4gnmZWF3tTxq1IkN
KVJuGdcPLg33zFAXzmoPOzjrTmr+1DUEahBbY/9oGcQQh0Ir9lTdd0Uym40FN/7jD
A8G1CeK8lsL+TZ5dTUBPCI2LLd2p7c3SddMNM/GUZdoJ4LXKx3JnDu8lYpjOcL21Q
HjqfNSmAHzX8skeryjawwi0yijJYwsyrB629ek0p/v16uTojs6JGddOmnz9z1/pBct
Rw6w83d0jNQc2yYg89xOl9q0Z7G6rThyNuJwworN5FaPJB2Pl7pHf2uJZEp0mq1S
N3Lcfre2yXig8TcrWjDY8k9VrEJG0G3n9FVv9hKvob9ngUMkmyxE5E4VWyab5gVt2
m0XXJjz5Sc3530dQ9SXhFS7s3060/yl0BBWnTtfu9zGdKaz4lWo25Q0r7HD5y/MwU
CbqpRVqXJxGHYd3PEYaXkdwhAi3EbarF7R8r3hKzYCpXxfI4=
-----END CERTIFICATE-----
"""
m2_x509_cert = M2CRYPTO_X509.load_cert_string(cert_pem_string)

local_ski_ext = m2_x509_cert.get_ext('subjectKeyIdentifier')
# Extract the value from the extension. It will be a string like '1C:E6:...'
local_ski_value = local_ski_ext.get_value()

# For authorityKeyIdentifier, use 'keyid:' followed by the subjectKeyIdentifier of the issuer.
# In this case, the issuer is the same certificate, so we use its own SKI.
aki_value = f"keyid:{local_ski_value}"

M2CRYPTO_X509.new_extension('subjectKeyIdentifier', sub_key_id)
print('Subject Key Identifier extension created successfully.')

# Now, create the authorityKeyIdentifier with the correctly formatted value
M2CRYPTO_X509.new_extension('authorityKeyIdentifier', aki_value)
print('Authority Key Identifier extension created successfully.')

Explanation of Changes:

  1. local_ski_ext = m2_x509_cert.get_ext('subjectKeyIdentifier'): We retrieve the subjectKeyIdentifier extension object from the loaded certificate.
  2. local_ski_value = local_ski_ext.get_value(): We extract the actual string value (e.g., '1C:E6:...') from the subjectKeyIdentifier extension.
  3. aki_value = f"keyid:{local_ski_value}": This is the crucial change. We construct the authorityKeyIdentifier string using the keyid: prefix followed by the retrieved subjectKeyIdentifier value. This is a valid format that OpenSSL expects.

By providing a valid string for the authorityKeyIdentifier extension, you avoid passing invalid data to OpenSSL's underlying functions, thereby preventing the segmentation fault.

~mcepl 14 hours ago

You're right, simply fixing that specific test case doesn't address the underlying robustness of M2Crypto. The core issue is that M2Crypto seems to be directly passing unvalidated Python strings to low-level OpenSSL C functions, which then causes a segmentation fault when those C functions receive data in an unexpected format.

To make M2Crypto more robust and prevent such crashes, the developers should implement stricter input validation within the M2Crypto layer before calling OpenSSL functions. Here's a breakdown of where and how they could do that:

#General Principles for Robustness in M2Crypto

  1. Input Type and Format Validation:

    • Strict Type Checking: Ensure that arguments passed to M2Crypto functions are of the expected Python types (e.g., str, bytes, int).
    • Format Validation (Regex, Specific Parsers): For string inputs that represent structured data (like key identifiers, dates, distinguished names), validate their format using regular expressions or dedicated parsing functions before passing them to OpenSSL.
    • Length Constraints: Check if string inputs adhere to reasonable length limits to prevent buffer overflows or excessive memory allocation in OpenSSL.

  2. Error Handling (Python vs. C):

    • Translate OpenSSL Errors: OpenSSL functions typically return error codes. M2Crypto should catch these, translate them into meaningful Python exceptions (e.g., ValueError, TypeError, OpenSSLError), and raise them instead of letting the program crash.
    • Pre-emptive Checks: If M2Crypto can anticipate a failure based on input format, it should raise a Python exception early, preventing the call to the OpenSSL C function altogether.

  3. Encapsulation and Abstraction:

    • Wrapper Functions: Instead of directly exposing raw OpenSSL function calls, M2Crypto should wrap them in Python functions that perform validation and error handling.
    • Data Structures: If OpenSSL expects complex C structures, M2Crypto should provide Pythonic ways to construct these, performing validation at the Python level.

#Critical Values and How to Validate Them (Focus on X509.new_extension)

The specific crash you observed occurred during M2CRYPTO_X509.new_extension('authorityKeyIdentifier', 'keyid'). Let's analyze the new_extension method and the types of extensions it handles:

The X509.new_extension(name, value, critical=0) function in M2Crypto is a wrapper around OpenSSL's X509V3_EXT_conf or similar functions. These functions expect the value parameter to be in a specific string format that OpenSSL's v3_conf module can parse.

Here are the critical values and validation strategies for various X.509 extensions that new_extension might encounter:

#1. subjectKeyIdentifier (NID_subject_key_identifier or 2.5.29.14)
  • Expected Format: A hexadecimal string, optionally colon-separated (e.g., '1C:E6:F0:...' or '1CE6F0...'). It represents a hash of the subject's public key.
  • Validation:
    • Type: Must be a string.
    • Characters: Should only contain hexadecimal characters (0-9, A-F, a-f) and optional colons (:).
    • Length: After removing colons, the length should be even and typically corresponds to a SHA-1 hash (40 characters for 20 bytes).
    • Example: 
      import re

def validate_subject_key_identifier(value):
    if not isinstance(value, str):
        raise TypeError("Subject Key Identifier must be a string.")
    cleaned_value = value.replace(':', '').strip()
    if not re.fullmatch(r'^[0-9a-fA-F]*$', cleaned_value):
        raise ValueError("Subject Key Identifier contains invalid characters (non-hex).")
    if len(cleaned_value) % 2 != 0:
        raise ValueError("Subject Key Identifier hex string has an odd length.")
    # Optional: Check for typical SHA-1 length
    # if len(cleaned_value) != 40:
    #     warnings.warn("Subject Key Identifier does not appear to be a SHA-1 hash length.")
#2. authorityKeyIdentifier (NID_authority_key_identifier or 2.5.29.35)

This is where your crash happened. The value for this extension is highly structured.

  • Expected Formats (as v3_conf strings):
    • keyid:<hex_string_of_ski>: Most common. Refers to the subjectKeyIdentifier of the issuer.
    • keyid:always: OpenSSL derives the keyid from the issuer's public key.
    • issuer:<DN_string>: Refers to the issuer's distinguished name (less common for AKI).
    • issuer:always: OpenSSL derives issuer information.
    • keyid,issuer:always: Combines keyid with issuer details.
    • keyid,serial:<hex_serial_number>: Refers to keyid and serial number.
  • Validation:
    • Type: Must be a string.
    • Keywords: Check for valid keywords like keyid, issuer, serial.
    • keyid format: If keyid: is present, the part after it must be a valid hex string (similar to subjectKeyIdentifier validation).
    • serial format: If serial: is present, the part after it must be a valid hex string representing the serial number.
    • Syntactic Structure: Use regular expressions or a simple parser to ensure the string adheres to the comma-separated key:value or key format expected by v3_conf.
    • Example (simplified for keyid case): 
      import re

def validate_authority_key_identifier(value):
    if not isinstance(value, str):
        raise TypeError("Authority Key Identifier must be a string.")

    if value.startswith("keyid:"):
        ski_part = value[len("keyid:"):]
        # Reuse subjectKeyIdentifier validation logic
        validate_subject_key_identifier(ski_part)
    elif value == "keyid:always" or value == "issuer:always" or value == "keyid,issuer:always":
        pass # These are valid literal values
    elif value.startswith("issuer:") or value.startswith("serial:"):
        # More complex validation needed for DNs or serials
        # For simplicity, we might just check for non-empty string here if we can't parse DNs
        pass
    else:
        raise ValueError(f"Invalid format for Authority Key Identifier: '{value}'. "
                         "Expected formats like 'keyid:<hex>', 'keyid:always', etc.")
      Self-correction: For authorityKeyIdentifier, it's not always just a hex string. It can involve keyid, issuer, and serial components. The validation needs to be robust enough to handle these combinations. OpenSSL's v3_conf parser is quite flexible, so M2Crypto could try to replicate some of that flexibility or enforce a stricter subset.
#3. basicConstraints (NID_basic_constraints or 2.5.29.19)
  • Expected Format: A string like 'CA:TRUE, pathlen:1' or 'CA:FALSE'.
  • Validation:
    • Type: Must be a string.
    • Keywords: Check for CA and pathlen.
    • Boolean Values: TRUE/FALSE for CA.
    • Integer Values: Non-negative integer for pathlen.
    • Syntactic Structure: Comma-separated key:value pairs.
#4. extendedKeyUsage (NID_ext_key_usage or 2.5.29.37)
  • Expected Format: Comma-separated list of OID names (e.g., 'serverAuth, clientAuth') or OIDs (e.g., '1.3.6.1.5.5.7.3.1, 1.3.6.1.5.5.7.3.2').
  • Validation:
    • Type: Must be a string.
    • Individual entries: Each entry should either be a known OID short name or a valid OID dotted-decimal string.
    • Separator: Comma.
#5. keyUsage (NID_key_usage or 2.5.29.15)
  • Expected Format: Comma-separated list of usage names (e.g., 'digitalSignature, keyEncipherment').
  • Validation:
    • Type: Must be a string.
    • Keywords: Validate against a known set of valid key usage flags (e.g., digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment, keyAgreement, keyCertSign, cRLSign, encipherOnly, decipherOnly).
#General new_extension Validation Logic

Since new_extension takes a name argument, M2Crypto could have a dictionary or a series of if/elif statements that map extension NIDs/names to specific validation functions.

# Inside M2Crypto's X509 module (conceptual)

_EXTENSION_VALIDATORS = {
    'subjectKeyIdentifier': validate_subject_key_identifier,
    'authorityKeyIdentifier': validate_authority_key_identifier,
    # Add other validators for common extensions
    # 'basicConstraints': validate_basic_constraints,
    # 'extendedKeyUsage': validate_extended_key_usage,
    # 'keyUsage': validate_key_usage,
}

def new_extension(name, value, critical=0):
    # 1. Basic type validation for name, value, critical
    if not isinstance(name, str):
        raise TypeError("Extension name must be a string.")
    if not isinstance(critical, (int, bool)):
        raise TypeError("Critical flag must be an integer or boolean.")

    # 2. Specific validation based on extension name
    validator = _EXTENSION_VALIDATORS.get(name)
    if validator:
        try:
            validator(value)
        except (TypeError, ValueError) as e:
            raise ValueError(f"Invalid value for extension '{name}': {e}") from e
    else:
        # For unknown extensions, M2Crypto could be more permissive but
        # still perform basic string type checks
        if not isinstance(value, str):
            raise TypeError(f"Value for unknown extension '{name}' must be a string.")

    # 3. Call OpenSSL C function (swig_x509v3_ext_conf in the backtrace)
    # This call should now receive a pre-validated string
    # try:
    #     _m2crypto_wrap.x509v3_ext_conf(None, ctx, name, value, critical)
    # except OpenSSLError as e: # Assuming M2Crypto catches OpenSSL errors
    #     raise M2CryptoError(f"OpenSSL error creating extension '{name}': {e}")

#Implementing in M2Crypto

To implement this effectively, the M2Crypto developers would need to:

  1. Identify all functions that take string inputs destined for OpenSSL parsing.
  2. For each such function, determine the expected format(s) for the OpenSSL C API.
  3. Implement Python-level validation routines (using regex, parsing logic, or checks against known values).
  4. Integrate these validation routines into the M2Crypto Python wrapper functions, ensuring that appropriate Python exceptions are raised for invalid inputs.
  5. Enhance OpenSSL error reporting: Ensure that any errors returned by OpenSSL C functions are properly caught and translated into Python exceptions, providing more context than a segmentation fault.

By adding these layers of validation, M2Crypto would become significantly more robust, failing gracefully with informative Python exceptions rather than crashing the entire application with a segmentation fault.

~mcepl 14 hours ago

Matěj Cepl referenced this ticket in commit 1059949.

Register here or Log in to comment, or comment via email.