Skip to content

gh-99108: Add HACL* Blake2 implementation to hashlib #119316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 64 commits into from
Aug 13, 2024
Merged

gh-99108: Add HACL* Blake2 implementation to hashlib #119316

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
81b42f2
Refresh HACL*; update modules accordingly; fix namespacing
msprotz Mar 25, 2024
9514fb4
make regen-sbom
msprotz Mar 25, 2024
b6b8ea9
wip
msprotz Apr 19, 2024
94bb7aa
Initial prototype of hashlib running atop HACL*'s blake2
msprotz Apr 19, 2024
e8a3060
All hashlib tests pass
msprotz Apr 19, 2024
3896560
Merge remote-tracking branch 'upstream/main' into hacl_blake2
msprotz Jun 24, 2024
d9e5568
Import more files, now to figure out the build system...
msprotz Jun 25, 2024
0d7a2ce
No need to repeat those flags in configure -- they are set in Setup.s…
msprotz Jul 18, 2024
b00af13
Try some AC stuff
msprotz Jul 18, 2024
cd3782a
Merge remote-tracking branch 'upstream/main' into hacl_blake2
msprotz Jul 18, 2024
36f7193
Every single AX_CHECK_COMPILE_FLAG returns false because PY_BUILTIN_H…
msprotz Jul 18, 2024
102dbbf
And another pyconfig.h flag for the second variant -- doing this prop…
msprotz Jul 18, 2024
a61aa04
Getting there with the build
msprotz Jul 18, 2024
11d7c76
Multiplex everything
msprotz Jul 18, 2024
9e0df3c
Defeat unused variable warning
msprotz Jul 18, 2024
d959d6d
We seem to be down to a linking problem
msprotz Jul 18, 2024
8958943
Making progress on autoconf / make
msprotz Jul 19, 2024
538401e
Refresh HACL*
msprotz Jul 19, 2024
cfc7053
Fix a few more typos, build now seems ok
msprotz Jul 19, 2024
d564d3a
Eliminate trailing whitespace when importing HACL*
msprotz Jul 19, 2024
505e8c1
No GNU-isms (hopefully)
msprotz Jul 19, 2024
cc70e6a
Fix some typos
msprotz Jul 19, 2024
4b9e84e
Update the namespacing mechanism, and write down in a comment how to …
msprotz Jul 19, 2024
314f962
NEWS
msprotz Jul 19, 2024
def45e4
Update SBOM
msprotz Jul 19, 2024
97ddafd
Remove GNU-isms
msprotz Jul 22, 2024
2abfcc1
Remove libb2 from SBOM (thanks @sethmlarson)
msprotz Jul 22, 2024
a49343b
Merge branch 'hacl_blake2' of pro.github.com:msprotz/cpython into hac…
msprotz Jul 22, 2024
49cdab5
Regenerate SBOM
msprotz Jul 22, 2024
eb4c3bd
Merge branch 'main' into hacl_blake2
msprotz Jul 22, 2024
cc91e72
Refresh HACL* again, for whitespace
msprotz Jul 22, 2024
873c446
Fix Windows build
msprotz Jul 23, 2024
e3112cc
Merge branch 'hacl_blake2' of pro.github.com:msprotz/cpython into hac…
msprotz Jul 23, 2024
cf70288
Update SBOM
msprotz Jul 23, 2024
50325c2
Deal with ARM64
msprotz Jul 23, 2024
48c4bd0
Match statements in opcode need more recent Python
msprotz Jul 23, 2024
59bcdb9
Fix Windows build
msprotz Jul 23, 2024
22ee262
make regen-sbom
msprotz Jul 23, 2024
550ca99
Try to fix C analysis
msprotz Jul 23, 2024
278bc6c
Tweak Windows build
msprotz Jul 23, 2024
578494a
Build tweaks
msprotz Jul 23, 2024
24ed962
Bring blake2 in line with other hash modules for Py_MOD_GIL_NOT_USED
msprotz Jul 23, 2024
e54bc8e
Move CPU flags detection to per-module state
msprotz Jul 24, 2024
abaf42b
Update Modules/blake2module.c
msprotz Jul 24, 2024
7495a88
Update Modules/blake2module.c
msprotz Jul 24, 2024
fce0cc2
Update Modules/blake2module.c
msprotz Jul 24, 2024
5610b25
Update Modules/blake2module.c
msprotz Jul 24, 2024
38233db
Address various review comments
msprotz Jul 24, 2024
e0e8636
Merge branch 'main' into hacl_blake2
msprotz Jul 24, 2024
b8b943c
Add test_blake2_update_over_4gb.
gpshead Aug 1, 2024
503fb34
Update Modules/blake2module.c
msprotz Aug 1, 2024
0b6dba1
Update Modules/blake2module.c
msprotz Aug 1, 2024
2eccf53
Update Modules/blake2module.c
msprotz Aug 1, 2024
04e9b45
Update Modules/blake2module.c
msprotz Aug 1, 2024
a616575
Add missing includes
msprotz Aug 1, 2024
af0d73b
Every single AX_CHECK_COMPILE_FLAG returns false because PY_BUILTIN_H…
msprotz Jul 18, 2024
f3f95a2
Merge branch 'main' into hacl_blake2
gpshead Aug 5, 2024
f9089fd
Fix configure.ac
msprotz Aug 5, 2024
15db7c1
Merge remote-tracking branch 'origin/double_define' into hacl_blake2
msprotz Aug 5, 2024
bb1cf00
Merge branch 'main' into hacl_blake2
msprotz Aug 6, 2024
07853a5
Merge branch 'main' into hacl_blake2
msprotz Aug 7, 2024
b09d92f
Keep the runtime CPU flags and static compilation flags in check
msprotz Aug 10, 2024
53dfa81
Merge branch 'main' into hacl_blake2
gpshead Aug 13, 2024
40374ac
Merge branch 'main' into hacl_blake2
gpshead Aug 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions Makefile.pre.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -825,7 +825,7 @@ coverage-lcov:
@ # remove 3rd party modules, system headers and internal files with
@ # debug, test or dummy functions.
@lcov $(COVERAGE_LCOV_OPTIONS) --remove $(COVERAGE_INFO) \
'*/Modules/_blake2/impl/*' \
'*/Modules/_hacl/*' \
'*/Modules/_ctypes/libffi*/*' \
'*/Modules/_decimal/libmpdec/*' \
'*/Modules/expat/*' \
Expand Down Expand Up @@ -855,7 +855,7 @@ coverage-report: regen-token regen-frozen

# Run "Argument Clinic" over all source files
.PHONY: clinic
clinic: check-clean-src $(srcdir)/Modules/_blake2/blake2s_impl.c
clinic: check-clean-src
$(PYTHON_FOR_REGEN) $(srcdir)/Tools/clinic/clinic.py --make --exclude Lib/test/clinic.test.c --srcdir $(srcdir)

.PHONY: clinic-tests
Expand Down Expand Up @@ -885,11 +885,6 @@ pybuilddir.txt: $(PYTHON_FOR_BUILD_DEPS)
exit 1 ; \
fi

# blake2s is auto-generated from blake2b
$(srcdir)/Modules/_blake2/blake2s_impl.c: $(srcdir)/Modules/_blake2/blake2b_impl.c $(srcdir)/Modules/_blake2/blake2b2s.py
$(PYTHON_FOR_REGEN) $(srcdir)/Modules/_blake2/blake2b2s.py
$(PYTHON_FOR_REGEN) $(srcdir)/Tools/clinic/clinic.py -f $@

# Build static library
$(LIBRARY): $(LIBRARY_OBJS)
-rm -f $@
Expand Down Expand Up @@ -3110,18 +3105,24 @@ MODULE_CMATH_DEPS=$(srcdir)/Modules/_math.h
MODULE_MATH_DEPS=$(srcdir)/Modules/_math.h
MODULE_PYEXPAT_DEPS=@LIBEXPAT_INTERNAL@
MODULE_UNICODEDATA_DEPS=$(srcdir)/Modules/unicodedata_db.h $(srcdir)/Modules/unicodename_db.h
MODULE__BLAKE2_DEPS=$(srcdir)/Modules/_blake2/impl/blake2-config.h $(srcdir)/Modules/_blake2/impl/blake2-impl.h $(srcdir)/Modules/_blake2/impl/blake2.h $(srcdir)/Modules/_blake2/impl/blake2b-load-sse2.h $(srcdir)/Modules/_blake2/impl/blake2b-load-sse41.h $(srcdir)/Modules/_blake2/impl/blake2b-ref.c $(srcdir)/Modules/_blake2/impl/blake2b-round.h $(srcdir)/Modules/_blake2/impl/blake2b.c $(srcdir)/Modules/_blake2/impl/blake2s-load-sse2.h $(srcdir)/Modules/_blake2/impl/blake2s-load-sse41.h $(srcdir)/Modules/_blake2/impl/blake2s-load-xop.h $(srcdir)/Modules/_blake2/impl/blake2s-ref.c $(srcdir)/Modules/_blake2/impl/blake2s-round.h $(srcdir)/Modules/_blake2/impl/blake2s.c $(srcdir)/Modules/_blake2/blake2module.h $(srcdir)/Modules/hashlib.h
MODULE__BLAKE2_DEPS=$(LIBHACL_HEADERS) \
Modules/_hacl/Hacl_Hash_Blake2b.h Modules/_hacl/Hacl_Hash_Blake2s.h \
Modules/_hacl/Hacl_Hash_Blake2b_Simd256.h Modules/_hacl/Hacl_Hash_Blake2s_Simd128.h \
Modules/_hacl/internal/Hacl_Hash_Blake2b.h Modules/_hacl/internal/Hacl_Hash_Blake2s.h \
Modules/_hacl/internal/Hacl_Hash_Blake2b_Simd256.h Modules/_hacl/internal/Hacl_Hash_Blake2s_Simd128.h \
Modules/_hacl/Hacl_Hash_Blake2b.c Modules/_hacl/Hacl_Hash_Blake2s.c \
Modules/_hacl/Hacl_Hash_Blake2b_Simd256.c Modules/_hacl/Hacl_Hash_Blake2s_Simd128.c
MODULE__CTYPES_DEPS=$(srcdir)/Modules/_ctypes/ctypes.h
MODULE__CTYPES_TEST_DEPS=$(srcdir)/Modules/_ctypes/_ctypes_test_generated.c.h
MODULE__CTYPES_MALLOC_CLOSURE=@MODULE__CTYPES_MALLOC_CLOSURE@
MODULE__DECIMAL_DEPS=$(srcdir)/Modules/_decimal/docstrings.h @LIBMPDEC_INTERNAL@
MODULE__ELEMENTTREE_DEPS=$(srcdir)/Modules/pyexpat.c @LIBEXPAT_INTERNAL@
MODULE__HASHLIB_DEPS=$(srcdir)/Modules/hashlib.h
MODULE__IO_DEPS=$(srcdir)/Modules/_io/_iomodule.h
MODULE__MD5_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_MD5.h Modules/_hacl/Hacl_Hash_MD5.c
MODULE__SHA1_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_SHA1.h Modules/_hacl/Hacl_Hash_SHA1.c
MODULE__MD5_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_MD5.h Modules/_hacl/internal/Hacl_Hash_MD5.h Modules/_hacl/Hacl_Hash_MD5.c
MODULE__SHA1_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_SHA1.h Modules/_hacl/internal/Hacl_Hash_SHA1.h Modules/_hacl/Hacl_Hash_SHA1.c
MODULE__SHA2_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_SHA2_HEADERS) $(LIBHACL_SHA2_A)
MODULE__SHA3_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_SHA3.h Modules/_hacl/Hacl_Hash_SHA3.c
MODULE__SHA3_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_HEADERS) Modules/_hacl/Hacl_Hash_SHA3.h Modules/_hacl/internal/Hacl_Hash_SHA3.h Modules/_hacl/Hacl_Hash_SHA3.c
MODULE__SOCKET_DEPS=$(srcdir)/Modules/socketmodule.h $(srcdir)/Modules/addrinfo.h $(srcdir)/Modules/getaddrinfo.c $(srcdir)/Modules/getnameinfo.c
MODULE__SSL_DEPS=$(srcdir)/Modules/_ssl.h $(srcdir)/Modules/_ssl/cert.c $(srcdir)/Modules/_ssl/debughelpers.c $(srcdir)/Modules/_ssl/misc.c $(srcdir)/Modules/_ssl_data_111.h $(srcdir)/Modules/_ssl_data_300.h $(srcdir)/Modules/socketmodule.h
MODULE__TESTCAPI_DEPS=$(srcdir)/Modules/_testcapi/parts.h $(srcdir)/Modules/_testcapi/util.h
Expand Down
2 changes: 1 addition & 1 deletion Modules/Setup.stdlib.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
@MODULE__SHA1_TRUE@_sha1 sha1module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_SHA1.c -D_BSD_SOURCE -D_DEFAULT_SOURCE
@MODULE__SHA2_TRUE@_sha2 sha2module.c -I$(srcdir)/Modules/_hacl/include Modules/_hacl/libHacl_Hash_SHA2.a
@MODULE__SHA3_TRUE@_sha3 sha3module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_SHA3.c -D_BSD_SOURCE -D_DEFAULT_SOURCE
@MODULE__BLAKE2_TRUE@_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
@MODULE__BLAKE2_TRUE@_blake2 blake2module.c -I$(srcdir)/Modules/_hacl/ -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_Blake2s.c _hacl/Hacl_Hash_Blake2b.c _hacl/Hacl_Hash_Blake2s_Simd128.c _hacl/Hacl_Hash_Blake2b_Simd256.c _hacl/Lib_Memzero0.c -D_BSD_SOURCE -D_DEFAULT_SOURCE

############################################################################
# XML and text
Expand Down
49 changes: 0 additions & 49 deletions Modules/_blake2/blake2b2s.py
View file
Open in desktop

This file was deleted.

Loading
Loading