File scanning: validate wheel digests against pypi.org #199

Oleg-Komarov · 2024-07-01T14:39:43+02:00

Oleg-Komarov commented

2024-07-01 14:39:43 +02:00

This PR adds a new check to background file scans:
wheel digests are verified using PyPI json API
https://warehouse.pypa.io/api-reference/json.html

This check should flag uploads that try to ship wheels
that are not published on PyPI, a file is flagged when either

it contains a wheel that is not on PyPI:
a combination of project name, version and platform tags is checked
at least one of the wheels that were found on PyPI has a mismatching sha256 digest

Although the fact that something is published on PyPI is not a guarantee
that the code is safe to load, this additional step should introduce at least
some barriers to uploading malicious code.

We can potentially improve on this further by e.g. integrating with
https://docs.virustotal.com/docs/api-overview

This PR adds a new check to background file scans: wheel digests are verified using PyPI json API https://warehouse.pypa.io/api-reference/json.html This check should flag uploads that try to ship wheels that are not published on PyPI, a file is flagged when either - it contains a wheel that is not on PyPI: a combination of project name, version and platform tags is checked - at least one of the wheels that were found on PyPI has a mismatching sha256 digest Although the fact that something is published on PyPI is not a guarantee that the code is safe to load, this additional step should introduce at least some barriers to uploading malicious code. We can potentially improve on this further by e.g. integrating with https://docs.virustotal.com/docs/api-overview

❤️ 1

Oleg-Komarov added 1 commit 2024-07-01 14:39:44 +02:00

wip ae60cdd3d3

Oleg-Komarov added 1 commit 2024-07-09 16:47:14 +02:00

Merge branch 'main' into validate-wheels 24f402bd4e

Oleg-Komarov added 1 commit 2024-07-09 17:08:00 +02:00

update wheel fetching code c6ef475c00

Oleg-Komarov added 2 commits 2024-07-09 19:46:04 +02:00

renames in test ad86cf5bec

add test for validate_wheels dc4fa3cef5

Oleg-Komarov changed title from ~~WIP: validate wheels~~ to File scanning: validate wheel digests against pypi.org

2024-07-09 19:47:04 +02:00