-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove dependency on hash-wasm #221
Labels
hub
@huggingface/hub related
Comments
coyotte508
changed the title
Remove dependency on hash-wasm for @huggingface/hub
Remove dependency on hash-wasm
Jul 14, 2023
Here's an example of Sha worker code with 0 dependencies: /**
* Adapted from https://github.com/Daninet/hash-wasm
*
* Copyright (c) 2020 Dani Biró, MIT license
*/
const rawWasm = {
name: 'sha256',
data: 'AGFzbQEAAAABEQRgAAF/YAF/AGACf38AYAAAAwgHAAEBAgMAAgQFAXABAQEFBAEBAgIGDgJ/AUHwiQULfwBBgAgLB3AIBm1lbW9yeQIADkhhc2hfR2V0QnVmZmVyAAAJSGFzaF9Jbml0AAELSGFzaF9VcGRhdGUAAgpIYXNoX0ZpbmFsAAQNSGFzaF9HZXRTdGF0ZQAFDkhhc2hfQ2FsY3VsYXRlAAYKU1RBVEVfU0laRQMBCuJIBwUAQYAJC50BAEEAQgA3A8CJAUEAQRxBICAAQeABRiIAGzYC6IkBQQBCp5/mp8b0k/2+f0Krs4/8kaOz8NsAIAAbNwPgiQFBAEKxloD+n6KFrOgAQv+kuYjFkdqCm38gABs3A9iJAUEAQpe6w4OTp5aHd0Ly5rvjo6f9p6V/IAAbNwPQiQFBAELYvZaI/KC1vjZC58yn0NbQ67O7fyAAGzcDyIkBC4ACAgF+Bn9BAEEAKQPAiQEiASAArXw3A8CJAQJAAkACQCABp0E/cSICDQBBgAkhAgwBCwJAIABBwAAgAmsiAyADIABLIgQbIgVFDQAgAkGAiQFqIQZBACECQQAhBwNAIAYgAmogAkGACWotAAA6AAAgBSAHQQFqIgdB/wFxIgJLDQALCyAEDQFByIkBQYCJARADIAAgA2shACADQYAJaiECCwJAIABBwABJDQADQEHIiQEgAhADIAJBwABqIQIgAEFAaiIAQT9LDQALCyAARQ0AQQAhB0EAIQUDQCAHQYCJAWogAiAHai0AADoAACAAIAVBAWoiBUH/AXEiB0sNAAsLC5M+AUV/IAAgASgCPCICQRh0IAJBCHRBgID8B3FyIAJBCHZBgP4DcSACQRh2cnIiAkEOdyACQQN2cyACQRl3cyABKAI4IgNBGHQgA0EIdEGAgPwHcXIgA0EIdkGA/gNxIANBGHZyciIDaiABKAIgIgRBGHQgBEEIdEGAgPwHcXIgBEEIdkGA/gNxIARBGHZyciIFQQ53IAVBA3ZzIAVBGXdzIAEoAhwiBEEYdCAEQQh0QYCA/AdxciAEQQh2QYD+A3EgBEEYdnJyIgZqIAEoAgQiBEEYdCAEQQh0QYCA/AdxciAEQQh2QYD+A3EgBEEYdnJyIgdBDncgB0EDdnMgB0EZd3MgASgCACIEQRh0IARBCHRBgID8B3FyIARBCHZBgP4DcSAEQRh2cnIiCGogASgCJCIEQRh0IARBCHRBgID8B3FyIARBCHZBgP4DcSAEQRh2cnIiCWogA0ENdyADQQp2cyADQQ93c2oiBGogASgCGCIKQRh0IApBCHRBgID8B3FyIApBCHZBgP4DcSAKQRh2cnIiC0EOdyALQQN2cyALQRl3cyABKAIUIgpBGHQgCkEIdEGAgPwHcXIgCkEIdkGA/gNxIApBGHZyciIMaiADaiABKAIQIgpBGHQgCkEIdEGAgPwHcXIgCkEIdkGA/gNxIApBGHZyciINQQ53IA1BA3ZzIA1BGXdzIAEoAgwiCkEYdCAKQQh0QYCA/AdxciAKQQh2QYD+A3EgCkEYdnJyIg5qIAEoAjAiCkEYdCAKQQh0QYCA/AdxciAKQQh2QYD+A3EgCkEYdnJyIg9qIAEoAggiCkEYdCAKQQh0QYCA/AdxciAKQQh2QYD+A3EgCkEYdnJyIhBBDncgEEEDdnMgEEEZd3MgB2ogASgCKCIKQRh0IApBCHRBgID8B3FyIApBCHZBgP4DcSAKQRh2cnIiEWogAkENdyACQQp2cyACQQ93c2oiCkENdyAKQQp2cyAKQQ93c2oiEkENdyASQQp2cyASQQ93c2oiE0ENdyATQQp2cyATQQ93c2oiFGogASgCNCIVQRh0IBVBCHRBgID8B3FyIBVBCHZBgP4DcSAVQRh2cnIiFkEOdyAWQQN2cyAWQRl3cyAPaiATaiABKAIsIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZyciIXQQ53IBdBA3ZzIBdBGXdzIBFqIBJqIAlBDncgCUEDdnMgCUEZd3MgBWogCmogBkEOdyAGQQN2cyAGQRl3cyALaiACaiAMQQ53IAxBA3ZzIAxBGXdzIA1qIBZqIA5BDncgDkEDdnMgDkEZd3MgEGogF2ogBEENdyAEQQp2cyAEQQ93c2oiFUENdyAVQQp2cyAVQQ93c2oiGEENdyAYQQp2cyAYQQ93c2oiGUENdyAZQQp2cyAZQQ93c2oiGkENdyAaQQp2cyAaQQ93c2oiG0ENdyAbQQp2cyAbQQ93c2oiHEENdyAcQQp2cyAcQQ93c2oiHUEOdyAdQQN2cyAdQRl3cyADQQ53IANBA3ZzIANBGXdzIBZqIBlqIA9BDncgD0EDdnMgD0EZd3MgF2ogGGogEUEOdyARQQN2cyARQRl3cyAJaiAVaiAUQQ13IBRBCnZzIBRBD3dzaiIeQQ13IB5BCnZzIB5BD3dzaiIfQQ13IB9BCnZzIB9BD3dzaiIgaiAUQQ53IBRBA3ZzIBRBGXdzIBlqIARBDncgBEEDdnMgBEEZd3MgAmogGmogIEENdyAgQQp2cyAgQQ93c2oiIWogE0EOdyATQQN2cyATQRl3cyAYaiAgaiASQQ53IBJBA3ZzIBJBGXdzIBVqIB9qIApBDncgCkEDdnMgCkEZd3MgBGogHmogHUENdyAdQQp2cyAdQQ93c2oiIkENdyAiQQp2cyAiQQ93c2oiI0ENdyAjQQp2cyAjQQ93c2oiJEENdyAkQQp2cyAkQQ93c2oiJWogHEEOdyAcQQN2cyAcQRl3cyAfaiAkaiAbQQ53IBtBA3ZzIBtBGXdzIB5qICNqIBpBDncgGkEDdnMgGkEZd3MgFGogImogGUEOdyAZQQN2cyAZQRl3cyATaiAdaiAYQQ53IBhBA3ZzIBhBGXdzIBJqIBxqIBVBDncgFUEDdnMgFUEZd3MgCmogG2ogIUENdyAhQQp2cyAhQQ93c2oiJkENdyAmQQp2cyAmQQ93c2oiJ0ENdyAnQQp2cyAnQQ93c2oiKEENdyAoQQp2cyAoQQ93c2oiKUENdyApQQp2cyApQQ93c2oiKkENdyAqQQp2cyAqQQ93c2oiK0ENdyArQQp2cyArQQ93c2oiLEEOdyAsQQN2cyAsQRl3cyAgQQ53ICBBA3ZzICBBGXdzIBxqIChqIB9BDncgH0EDdnMgH0EZd3MgG2ogJ2ogHkEOdyAeQQN2cyAeQRl3cyAaaiAmaiAlQQ13ICVBCnZzICVBD3dzaiItQQ13IC1BCnZzIC1BD3dzaiIuQQ13IC5BCnZzIC5BD3dzaiIvaiAlQQ53ICVBA3ZzICVBGXdzIChqICFBDncgIUEDdnMgIUEZd3MgHWogKWogL0ENdyAvQQp2cyAvQQ93c2oiMGogJEEOdyAkQQN2cyAkQRl3cyAnaiAvaiAjQQ53ICNBA3ZzICNBGXdzICZqIC5qICJBDncgIkEDdnMgIkEZd3MgIWogLWogLEENdyAsQQp2cyAsQQ93c2oiMUENdyAxQQp2cyAxQQ93c2oiMkENdyAyQQp2cyAyQQ93c2oiM0ENdyAzQQp2cyAzQQ93c2oiNGogK0EOdyArQQN2cyArQRl3cyAuaiAzaiAqQQ53ICpBA3ZzICpBGXdzIC1qIDJqIClBDncgKUEDdnMgKUEZd3MgJWogMWogKEEOdyAoQQN2cyAoQRl3cyAkaiAsaiAnQQ53ICdBA3ZzICdBGXdzICNqICtqICZBDncgJkEDdnMgJkEZd3MgImogKmogMEENdyAwQQp2cyAwQQ93c2oiNUENdyA1QQp2cyA1QQ93c2oiNkENdyA2QQp2cyA2QQ93c2oiN0ENdyA3QQp2cyA3QQ93c2oiOEENdyA4QQp2cyA4QQ93c2oiOUENdyA5QQp2cyA5QQ93c2oiOkENdyA6QQp2cyA6QQ93c2oiOyA5IDEgKyApICcgISAfIBQgEiACIBcgBiAAKAIQIjwgDmogACgCFCI9IBBqIAAoAhgiPiAHaiAAKAIcIj8gPEEadyA8QRV3cyA8QQd3c2ogPiA9cyA8cSA+c2ogCGpBmN+olARqIkAgACgCDCJBaiIHID0gPHNxID1zaiAHQRp3IAdBFXdzIAdBB3dzakGRid2JB2oiQiAAKAIIIkNqIg4gByA8c3EgPHNqIA5BGncgDkEVd3MgDkEHd3NqQc/3g657aiJEIAAoAgQiRWoiECAOIAdzcSAHc2ogEEEadyAQQRV3cyAQQQd3c2pBpbfXzX5qIkYgACgCACIBaiIIaiALIBBqIAwgDmogByANaiAIIBAgDnNxIA5zaiAIQRp3IAhBFXdzIAhBB3dzakHbhNvKA2oiDSBDIEUgAXNxIEUgAXFzIAFBHncgAUETd3MgAUEKd3NqIEBqIgdqIgYgCCAQc3EgEHNqIAZBGncgBkEVd3MgBkEHd3NqQfGjxM8FaiJAIAdBHncgB0ETd3MgB0EKd3MgByABcyBFcSAHIAFxc2ogQmoiDmoiCyAGIAhzcSAIc2ogC0EadyALQRV3cyALQQd3c2pBpIX+kXlqIkIgDkEedyAOQRN3cyAOQQp3cyAOIAdzIAFxIA4gB3FzaiBEaiIQaiIIIAsgBnNxIAZzaiAIQRp3IAhBFXdzIAhBB3dzakHVvfHYemoiRCAQQR53IBBBE3dzIBBBCndzIBAgDnMgB3EgECAOcXNqIEZqIgdqIgxqIBEgCGogCSALaiAFIAZqIAwgCCALc3EgC3NqIAxBGncgDEEVd3MgDEEHd3NqQZjVnsB9aiIJIAdBHncgB0ETd3MgB0EKd3MgByAQcyAOcSAHIBBxc2ogDWoiDmoiBiAMIAhzcSAIc2ogBkEadyAGQRV3cyAGQQd3c2pBgbaNlAFqIhEgDkEedyAOQRN3cyAOQQp3cyAOIAdzIBBxIA4gB3FzaiBAaiIQaiIIIAYgDHNxIAxzaiAIQRp3IAhBFXdzIAhBB3dzakG+i8ahAmoiFyAQQR53IBBBE3dzIBBBCndzIBAgDnMgB3EgECAOcXNqIEJqIgdqIgsgCCAGc3EgBnNqIAtBGncgC0EVd3MgC0EHd3NqQcP7sagFaiIFIAdBHncgB0ETd3MgB0EKd3MgByAQcyAOcSAHIBBxc2ogRGoiDmoiDGogAyALaiAWIAhqIA8gBmogDCALIAhzcSAIc2ogDEEadyAMQRV3cyAMQQd3c2pB9Lr5lQdqIg8gDkEedyAOQRN3cyAOQQp3cyAOIAdzIBBxIA4gB3FzaiAJaiICaiIQIAwgC3NxIAtzaiAQQRp3IBBBFXdzIBBBB3dzakH+4/qGeGoiCyACQR53IAJBE3dzIAJBCndzIAIgDnMgB3EgAiAOcXNqIBFqIgNqIgggECAMc3EgDHNqIAhBGncgCEEVd3MgCEEHd3NqQaeN8N55aiIMIANBHncgA0ETd3MgA0EKd3MgAyACcyAOcSADIAJxc2ogF2oiB2oiDiAIIBBzcSAQc2ogDkEadyAOQRV3cyAOQQd3c2pB9OLvjHxqIgkgB0EedyAHQRN3cyAHQQp3cyAHIANzIAJxIAcgA3FzaiAFaiICaiIGaiAVIA5qIAogCGogBiAOIAhzcSAIcyAQaiAEaiAGQRp3IAZBFXdzIAZBB3dzakHB0+2kfmoiECACQR53IAJBE3dzIAJBCndzIAIgB3MgA3EgAiAHcXNqIA9qIgNqIgogBiAOc3EgDnNqIApBGncgCkEVd3MgCkEHd3NqQYaP+f1+aiIOIANBHncgA0ETd3MgA0EKd3MgAyACcyAHcSADIAJxc2ogC2oiBGoiEiAKIAZzcSAGc2ogEkEadyASQRV3cyASQQd3c2pBxruG/gBqIgggBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAMaiICaiIVIBIgCnNxIApzaiAVQRp3IBVBFXdzIBVBB3dzakHMw7KgAmoiBiACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIAlqIgNqIgdqIBkgFWogEyASaiAKIBhqIAcgFSASc3EgEnNqIAdBGncgB0EVd3MgB0EHd3NqQe/YpO8CaiIYIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogEGoiBGoiCiAHIBVzcSAVc2ogCkEadyAKQRV3cyAKQQd3c2pBqonS0wRqIhUgBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAOaiICaiISIAogB3NxIAdzaiASQRp3IBJBFXdzIBJBB3dzakHc08LlBWoiGSACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIAhqIgNqIhMgEiAKc3EgCnNqIBNBGncgE0EVd3MgE0EHd3NqQdqR5rcHaiIHIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogBmoiBGoiFGogGyATaiAeIBJqIBogCmogFCATIBJzcSASc2ogFEEadyAUQRV3cyAUQQd3c2pB0qL5wXlqIhogBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAYaiICaiIKIBQgE3NxIBNzaiAKQRp3IApBFXdzIApBB3dzakHtjMfBemoiGCACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBVqIgNqIhIgCiAUc3EgFHNqIBJBGncgEkEVd3MgEkEHd3NqQcjPjIB7aiIVIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogGWoiBGoiEyASIApzcSAKc2ogE0EadyATQRV3cyATQQd3c2pBx//l+ntqIhkgBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAHaiICaiIUaiAdIBNqICAgEmogHCAKaiAUIBMgEnNxIBJzaiAUQRp3IBRBFXdzIBRBB3dzakHzl4C3fGoiGyACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBpqIgNqIgogFCATc3EgE3NqIApBGncgCkEVd3MgCkEHd3NqQceinq19aiIaIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogGGoiBGoiEiAKIBRzcSAUc2ogEkEadyASQRV3cyASQQd3c2pB0capNmoiGCAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBVqIgJqIhMgEiAKc3EgCnNqIBNBGncgE0EVd3MgE0EHd3NqQefSpKEBaiIVIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogGWoiA2oiFGogIyATaiAmIBJqIBQgEyASc3EgEnMgCmogImogFEEadyAUQRV3cyAUQQd3c2pBhZXcvQJqIhkgA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAbaiIEaiIKIBQgE3NxIBNzaiAKQRp3IApBFXdzIApBB3dzakG4wuzwAmoiGyAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBpqIgJqIhIgCiAUc3EgFHNqIBJBGncgEkEVd3MgEkEHd3NqQfzbsekEaiIaIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogGGoiA2oiEyASIApzcSAKc2ogE0EadyATQRV3cyATQQd3c2pBk5rgmQVqIhggA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAVaiIEaiIUaiAlIBNqICggEmogCiAkaiAUIBMgEnNxIBJzaiAUQRp3IBRBFXdzIBRBB3dzakHU5qmoBmoiFSAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBlqIgJqIgogFCATc3EgE3NqIApBGncgCkEVd3MgCkEHd3NqQbuVqLMHaiIZIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogG2oiA2oiEiAKIBRzcSAUc2ogEkEadyASQRV3cyASQQd3c2pBrpKLjnhqIhsgA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAaaiIEaiITIBIgCnNxIApzaiATQRp3IBNBFXdzIBNBB3dzakGF2ciTeWoiGiAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBhqIgJqIhRqIC4gE2ogKiASaiAtIApqIBQgEyASc3EgEnNqIBRBGncgFEEVd3MgFEEHd3NqQaHR/5V6aiIYIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogFWoiA2oiCiAUIBNzcSATc2ogCkEadyAKQRV3cyAKQQd3c2pBy8zpwHpqIhUgA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAZaiIEaiISIAogFHNxIBRzaiASQRp3IBJBFXdzIBJBB3dzakHwlq6SfGoiGSAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBtqIgJqIhMgEiAKc3EgCnNqIBNBGncgE0EVd3MgE0EHd3NqQaOjsbt8aiIbIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogGmoiA2oiFGogMCATaiAsIBJqIC8gCmogFCATIBJzcSASc2ogFEEadyAUQRV3cyAUQQd3c2pBmdDLjH1qIhogA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAYaiIEaiIKIBQgE3NxIBNzaiAKQRp3IApBFXdzIApBB3dzakGkjOS0fWoiGCAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBVqIgJqIhIgCiAUc3EgFHNqIBJBGncgEkEVd3MgEkEHd3NqQYXruKB/aiIVIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogGWoiA2oiEyASIApzcSAKc2ogE0EadyATQRV3cyATQQd3c2pB8MCqgwFqIhkgA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAbaiIEaiIUIBMgEnNxIBJzIApqIDVqIBRBGncgFEEVd3MgFEEHd3NqQZaCk80BaiIbIARBHncgBEETd3MgBEEKd3MgBCADcyACcSAEIANxc2ogGmoiAmoiCiA3aiAzIBRqIDYgE2ogMiASaiAKIBQgE3NxIBNzaiAKQRp3IApBFXdzIApBB3dzakGI2N3xAWoiGiACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBhqIgNqIhIgCiAUc3EgFHNqIBJBGncgEkEVd3MgEkEHd3NqQczuoboCaiIcIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogFWoiBGoiEyASIApzcSAKc2ogE0EadyATQRV3cyATQQd3c2pBtfnCpQNqIhUgBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAZaiICaiIKIBMgEnNxIBJzaiAKQRp3IApBFXdzIApBB3dzakGzmfDIA2oiGSACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBtqIgNqIhRqIC1BDncgLUEDdnMgLUEZd3MgKWogNWogNEENdyA0QQp2cyA0QQ93c2oiGCAKaiA4IBNqIDQgEmogFCAKIBNzcSATc2ogFEEadyAUQRV3cyAUQQd3c2pBytTi9gRqIhsgA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAaaiIEaiISIBQgCnNxIApzaiASQRp3IBJBFXdzIBJBB3dzakHPlPPcBWoiGiAEQR53IARBE3dzIARBCndzIAQgA3MgAnEgBCADcXNqIBxqIgJqIgogEiAUc3EgFHNqIApBGncgCkEVd3MgCkEHd3NqQfPfucEGaiIcIAJBHncgAkETd3MgAkEKd3MgAiAEcyADcSACIARxc2ogFWoiA2oiEyAKIBJzcSASc2ogE0EadyATQRV3cyATQQd3c2pB7oW+pAdqIh0gA0EedyADQRN3cyADQQp3cyADIAJzIARxIAMgAnFzaiAZaiIEaiIUaiAvQQ53IC9BA3ZzIC9BGXdzICtqIDdqIC5BDncgLkEDdnMgLkEZd3MgKmogNmogGEENdyAYQQp2cyAYQQ93c2oiFUENdyAVQQp2cyAVQQ93c2oiGSATaiA6IApqIBUgEmogFCATIApzcSAKc2ogFEEadyAUQRV3cyAUQQd3c2pB78aVxQdqIgogBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAbaiICaiISIBQgE3NxIBNzaiASQRp3IBJBFXdzIBJBB3dzakGU8KGmeGoiGyACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBpqIgNqIhMgEiAUc3EgFHNqIBNBGncgE0EVd3MgE0EHd3NqQYiEnOZ4aiIaIANBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogHGoiBGoiFCATIBJzcSASc2ogFEEadyAUQRV3cyAUQQd3c2pB+v/7hXlqIhwgBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAdaiICaiIVID9qNgIcIAAgQSACQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIApqIgNBHncgA0ETd3MgA0EKd3MgAyACcyAEcSADIAJxc2ogG2oiBEEedyAEQRN3cyAEQQp3cyAEIANzIAJxIAQgA3FzaiAaaiICQR53IAJBE3dzIAJBCndzIAIgBHMgA3EgAiAEcXNqIBxqIgpqNgIMIAAgPiAwQQ53IDBBA3ZzIDBBGXdzICxqIDhqIBlBDXcgGUEKdnMgGUEPd3NqIhkgEmogFSAUIBNzcSATc2ogFUEadyAVQRV3cyAVQQd3c2pB69nBonpqIhogA2oiEmo2AhggACBDIApBHncgCkETd3MgCkEKd3MgCiACcyAEcSAKIAJxc2ogGmoiA2o2AgggACA9IDFBDncgMUEDdnMgMUEZd3MgMGogGGogO0ENdyA7QQp2cyA7QQ93c2ogE2ogEiAVIBRzcSAUc2ogEkEadyASQRV3cyASQQd3c2pB98fm93tqIhggBGoiE2o2AhQgACBFIANBHncgA0ETd3MgA0EKd3MgAyAKcyACcSADIApxc2ogGGoiBGo2AgQgACA8IDVBDncgNUEDdnMgNUEZd3MgMWogOWogGUENdyAZQQp2cyAZQQ93c2ogFGogEyASIBVzcSAVc2ogE0EadyATQRV3cyATQQd3c2pB8vHFs3xqIhIgAmpqNgIQIAAgASAEQR53IARBE3dzIARBCndzIAQgA3MgCnEgBCADcXNqIBJqajYCAAv3BQIBfgR/QQApA8CJASIApyIBQQJ2QQ9xIgJBAnRBgIkBaiIDIAMoAgBBfyABQQN0IgFBGHEiA3RBf3NxQYABIAN0czYCAAJAAkACQCACQQ5JDQACQCACQQ5HDQBBAEEANgK8iQELQciJAUGAiQEQA0EAIQEMAQsgAkENRg0BIAJBAWohAQsgAUECdCEBA0AgAUGAiQFqQQA2AgAgAUEEaiIBQThHDQALQQApA8CJASIAp0EDdCEBC0EAIAFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYCvIkBQQAgAEIdiKciAUEYdCABQQh0QYCA/AdxciABQQh2QYD+A3EgAUEYdnJyNgK4iQFByIkBQYCJARADQQBBACgC5IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC5IkBQQBBACgC4IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC4IkBQQBBACgC3IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC3IkBQQBBACgC2IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC2IkBQQBBACgC1IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC1IkBQQBBACgC0IkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYC0IkBQQBBACgCzIkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZycjYCzIkBQQBBACgCyIkBIgFBGHQgAUEIdEGAgPwHcXIgAUEIdkGA/gNxIAFBGHZyciIBNgLIiQECQEEAKALoiQEiBEUNAEEAIAE6AIAJIARBAUYNACABQQh2IQNBASEBQQEhAgNAIAFBgAlqIAM6AAAgBCACQQFqIgJB/wFxIgFNDQEgAUHIiQFqLQAAIQMMAAsLCwYAQYCJAQujAQBBAEIANwPAiQFBAEEcQSAgAUHgAUYiARs2AuiJAUEAQqef5qfG9JP9vn9Cq7OP/JGjs/DbACABGzcD4IkBQQBCsZaA/p+ihazoAEL/pLmIxZHagpt/IAEbNwPYiQFBAEKXusODk6eWh3dC8ua746On/aelfyABGzcD0IkBQQBC2L2WiPygtb42QufMp9DW0Ouzu38gARs3A8iJASAAEAIQBAsLCwEAQYAICwRwAAAA',
hash: '817d957e',
};
export const MAX_HEAP = 16 * 1024;
const hashLength = 32;
async function base64toUint8Array(base64Str: string) {
const res = await fetch('data:application/octet-stream;base64,' + base64Str);
return new Uint8Array(await res.arrayBuffer());
}
if (typeof WebAssembly === 'undefined') {
throw new Error('WebAssembly is not supported in this environment!');
}
const init = async () => {
const wasmSource = await base64toUint8Array(rawWasm.data);
const compiledWasm = await WebAssembly.compile(wasmSource);
const wasmInstance = await WebAssembly.instantiate(compiledWasm);
const arrayOffset: number = (wasmInstance.exports.Hash_GetBuffer as any)();
const memoryBuffer = (wasmInstance.exports.memory as any).buffer;
const memoryView = new Uint8Array(memoryBuffer, arrayOffset, MAX_HEAP);
return {
wasmInstance,
memoryView,
};
};
const promise = init();
let ongoing = false;
self.addEventListener('message', async (event) => {
try {
if (ongoing) {
throw new Error('Hashing is already in progress');
}
const { file } = event.data as { file: File };
const { wasmInstance, memoryView } = await promise;
(wasmInstance.exports.Hash_Init as any)();
const reader = file.stream().getReader();
const total = file.size;
let bytesDone = 0;
while (true) {
const { done, value } = await reader.read();
if (done) {
break;
}
if (value) {
let read = 0;
while (read < value.length) {
const chunk = value.subarray(read, read + MAX_HEAP);
read += chunk.length;
memoryView.set(chunk);
(wasmInstance.exports.Hash_Update as any)(chunk.length);
}
}
bytesDone += value.length;
postMessage({ progress: bytesDone / total });
}
(wasmInstance.exports.Hash_Final as any)(null);
const hexChars = Array.from(memoryView.slice(0, hashLength))
.map((i) => i.toString(16).padStart(2, '0'))
.join('');
postMessage({ sha256: hexChars });
} catch (err) {
// Trick to bubble an error from a promise callback
// https://stackoverflow.com/a/40081158/835629
setTimeout(() => {
throw err;
});
} finally {
ongoing = false;
}
}); |
8 tasks
coyotte508
added a commit
that referenced
this issue
May 21, 2024
Resolves #221 In this PR, I moved source code of `hash-wasm` into vendor directory (c source code), also add compile script using emscripten. As a bonus, SIMD is also enabled, which should allow better performance. I also added `sha256-wrapper.ts` as a thin wrapper for the compiled wasm module. - `createSHA256` create the wasm module, works just like before. - `createSHA256WorkerCode` this function use `Function.toString()` trick to copy the module code into worker. This trick is already used in one of my project, [wllama](https://github.com/ngxson/wllama), so it should work on Firefox/Chrome/Safari (but feel free to test it further) --------- Co-authored-by: Eliott C <coyotte508@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
For example https://gist.github.com/coyotte508/3cfa9b011e819bcb4ba9286798c7d33c is the code of a worker, using
hash-wasm
's internals, but in a standalone file.It will improve bundled size too, but the main draw is having 0 dependencies.
The text was updated successfully, but these errors were encountered: